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ABSTRACT 


The  course  loading  forecasts  for  the  Operations  Research 
Department  published  by  the  Naval  Postgraduate  School 
Programming  Office  are  investigated  for  measures  of 
predictive  accuracy  by  course  and  curricula-  Both  course 
and  curricula  forecasts  are  categorized  as  problem  causing, 
reasonably  accurate,  or  undefined  due  to  limited  informa- 
tion. The  paucity  of  the  data  precludes  the  use  of  sophis- 
ticated statistical  techniques  to  identify  any  permanence  in 
effects  thereby  limiting  results  to  indications  of 
predictive  performance.  A  management  aid  is  developed 
consisting  of  a  data  tase  and  manipulative  APL  programs  for 
the  purposes  of  updating  and  displaying  current  information 
while  identifying  possible  synergistic  interactions  between 
data  subcatagories.  Appendix  A  contains  the  Users  Guide  for 
this  management  aid.  These  results  can  be  applied  to  the 
current  forecast  to  aid  the  department  chairman  in  creating 
the  faculty  teaching  schedule.  
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I.  BACKGROUND 

A-   PROBIEM  HISTORY 

The  Naval  Postgraduate  School  uses  a  complex  prediction 
system  to  forecast  expected  student  demand  for  courses  over 
the  upcoming  year.  This  forecast  is  computed  using  various 
inputs  from  several  sources  during  the  summer  quarter  (July 
to  September)  and  covers  the  following  academic  year  -  fall 
through  summer  quarters  inclusive.  The  tabled  output  of 
course  loading  predictions  so  computed  is  used  by  the 
various  Department  Chairmen  to  schedule  faculty  teaching 
responsibilities  and  research  time  for  that  year.  As  with 
any  forecasting  system,  this  set  of  course  loading  predici- 
tons  is  not  exact.  Various  errors  are  introduced  either 
from  the  simplif icaticn  of  the  process  required  to  get  the 
model  to  a  managable  size  or  from  erroneous  input.  The 
Operations  Research  Department  Chairman  noticed  an  inaccu- 
racy problem  when  a  class  with  a  predicted  size  of  approxi- 
mately 30  was  scheduled  and  approximately  75  students  showed 
up,  causing  a  disruption  of  faculty  teaching  plans. 
Believing  that  a  problem  existed,  he  posted  a  request  to 
research  the  scope  of  the  inaccuracies  as  pertaining  to  the 
courses  taught  by  the  Operations  Research  Department.  The 
study's  goal  would  be  to  determine  if  a  significant  problem 
existed  and,  if  so,  to  provide  either  a  series  of  adjust- 
ments to  be  made  tc  the  present  predictions  or  a  better 
prediction  method. 

B.   PRESENT  SYSTEM 

Until  1972,  the  scheduling  and  course  prediction  system 
was  run   by  hand  using   "corporate  memory";    the  historical 
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demand  for  courses  as  recalled  by  the  curriculum  representa- 
tives involved.  These  recollections  were  used  to  prepare  a 
proposed  schedule  that  the  students  and  curricular  offices 
used  and  modified  as  required.  By  1972,  this  system  had 
just  about  reached  tte  limits  of  its  capabilities.  At  that 
time,  the  scheduling  problem  was  passed  on  to  a  newly  hired 
NPS  computer  program  designer1  who  tailored  a  predictive 
computer  model  to  the  school's  needs.  That  model,  with 
modifications,  is  still  in  use  today. 

1 .   Scheduling  /  Prediction  Problems 

The  problem   faced  in  1971  is   the  same  today   -  How 
can  we  accurately  predict  three  items: 

•  The  courses  reguired  to  match  student  demand 

•  When  those  courses  should  be  taught 

•  How  many  students  will  attend 

Two  types  of  student  inputs  are  available;  those  who  are  not 
yet  onboard  (with  or  without  orders) ,  and  those  presently 
attending  the  school  in  any  given  capacity.  Both  types  of 
students  are  designated  as  attending  a  given  curriculum,  and 
from  that  designation  a  standard  program  of  courses  Is 
assigned.  This  assignment  leads  to  the  course  scheduling 
output.  Since  the  two  input  groups  are  significantly 
different  with  respect  to  treatment  within  the  model  itself, 
they  will  be  separately  discussed- 

a.   Students  not  yet  onboard 

The  Office  of  the  Chief  of  Naval  Operations 
publishes  a  yearly  gucta  plan,  [Ref-  1  ],  which  directs  the 
Commander,  Naval  Military  Personnel  Command  to  provide  the 
officer  personnel  to  fill  the  allocated  graduate  education 
billets.    The   fiscal  year  1985   guota  summary  is   shown  in 


1Mike  Troian.  Special  Assistant  for  Programs:   presently 
working  under  Code  0  13:  J.  Barron,  Capt.  USN  (ret) 
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table  I  .  These  quotas  are  further  broken  down  to  show,  by 
professional  designator  (i.e.  Surface  Warfare,  NFO, 
Cryptology  .  .  .),  what  curricula  the  officers  will  attend. 
Historically,  however,  these  quotas  have  been  guidelines  and 
not  strictly  followed.  Additionally,  Naval  Military 
Personnel  Command  (NMfC)  gives  only  a  rough  idea  of  in  which 
quarter  an  officer  will  arrive  during  a  given  year.  This  in 
turn  drives  the  input  problem  experienced  at  NPS.  Through 
liason  with  NMPC,  the  school  identifies  as  specifically  as 
possible  just  who  has  orders  to  attend  NPS  either  in  hand  or 
being  written.  Quota  shortfalls  tend  to  occur  in  the  "tech- 
nical" curricula  due  to  a  lack  of  available,  qualified 
personnel.  Conversly,  surpluses  occur  in  the  "non- 
technical" curricula.  As  with  any  military  organization, 
operational  commitments  such  as  maintaining  ships  in  the 
Eastern  Mediterranian  Sea  or  Indian  Ocean  adversly  affect 
the  arrival  of  Surface  Warfare  Officers  (who  consitute  the 
largest  group  in  the  NPS  student  population) . 

The  school  folds  all  of  this  quota  information 
and  experience  into  a  "best  guess"  of  the  number  of  students 
arriving  per  curriculum  per  quarter.2  As  this  process  is 
being  completed,  the  curricular  officers  designate  the  paths 
these  new  arrivals  will  follow.  This  is  done  either  by 
service  (i.e.  USN  students  take  the  following  courses:  . 
.)  or  by  past  experience  (i.e.  28%  of  incoming  students  need 
refresher  math  followed  by  •  .  .).  The  resulting  linking  of 
numbers  of  students  and  assigned  courses  provides  one  type 
of  input  to  the  scheduling  model. 


Additionally,  the  programming  office  contacts  indi- 
vidual service  and  foreign  representatives  to  further  refine 
the  expected  inputs  (since  [Ref.  1]  pertains  only  to  USN 
officers).  These  organizations  provide  less  reliable  data 
mainly  because  the  fiscal  year  for  other  US  services  starts 
in  January.  The  planning  for  their  personnel  transfers  is 
therefore  not  complete  until  October  or  November. 
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t.   Students  presently  attending  NPS 

The  scheduling  process  for  those  students 
already  onboard  is  somewhat  simpler.  Since  they  are  already 
designated  as  attending  a  given  curriculum  over  a  known 
period  of  time,  the  cnly  area  in  which  significant  problems 
are  found  is  that  of  elective  scheduling.  Since  the 
prediction  is  completed  a  full  year  in  advance,  out-guarter 
electives  may  not  have  been  correctly  selected  by  the 
student.  His  interests  or  his  required  course  load  may 
change,  thereby  changing  any  electives  he  has  selected. 
Additionally,  the  course  demand  may  be  such  that  the  course 
is  withdrawn  from  the  list  of  available  courses.  In  all 
cases,  however,  course  selection  input  covering  the  upcoming 
year  is  made  by  the  student  through  his  curricular  officer 
to  the  scheduling  office. 

2.   Scheduling  /  Prediction  System  Ose 

Once  elective  selections  have  been  made  by  the 
onboard  students  and  the  numbers  of  new  inputs  per  curric- 
ulum are  entered,  the  course  loading  prediction  model 
applies  standard  course  matrixes  to  generate  the  overall 
student  course  demand.  The  scheduling  system  is  demand 
based,  meaning  this  derived  demand  drives  the  faculty  hiring 
and  scheduling  process.  The  budget  process  also  is  based  on 
the  course  demand.  Teaching  funds  are  allocated  for  all 
instructors  teaching  course  sections  with  five  or  more 
students.  The  individual  department  chairmen  receive  a 
listing  of  student  demands  by  course  and  quarter  from  which 
they  create  a  proposed  schedule  of  courses  to  be  offered. 
This  schedule  is  further  adjusted  through  the  allocation  of 
teaching  funds  in  a  process  consisting  of  an  approximately 
month  long  dialog  between  the  department  chairmen,  division 
deans,  and  administrative  representatives  from  the 
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programming  office.  The  goal  of  this  process  is  to  create  an 
acceptable  schedule  of  courses  to  be  offered  balanced 
between  annual  operating  budget  funds,  student  course 
demands  and  faculty  research  requirements.  Through  this 
scheduling  process,  some  courses  are  dropped  and  others 
added,  creating  some  student  movement  among  the  courses 
offered.  The  goal  of  the  course  loading  prediction  model  is 
to  minimize  the  course  changes  caused  by  this  scheduling. 3 


3Mike  Troian,  the  Special  Assistant  for  Programs  working 
from  Ingersol  Room  375,  has  routinely  conducted  accuracy 
reviews  of  the  course  loading  prediction  model.  For  FY1983, 
this  review   has  shown  a   90%  accuracy  in   predicting  actual 


.jlty  reg_ 
reportedly  excellent 
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II-  INITIAL  STEPS  AT  ANALYSIS 

Before  any  fact-based  discussion  of  "better"  predictions 
could  te  started,  the  problem  facing  the  Department  Chairman 
had  to  be  defined.  A  single  historical  observation  of  a 
severe  prediction  error  that  caused  problems  is  not  signifi- 
cant considering  the  Eumber  of  predictions  and  observations 
made.  Historical  data  had  to  be  checked  to  determine  if  the 
problem  cf  inaccurate  predictions  actually  existed  or  was 
merely  assumed  based  on  a  seldom  occurring  event.  Since 
predictions  were  made  by  course,  the  obvious  choice  for 
comparison  was  the  historical  course  data  available  from  the 
registrar's  office.  Not  until  these  steps  were  completed 
could  a  search  for  a  tetter  method  of  prediction  be  started. 

A.   EATA  OEGAHIZATIOH 

One  of  the  initial  steps  of  the  analysis  was  an  organi- 
zation of  all  the  usable  data  into  a  quantifiable  format. 
Useful  information  available  consisted  of  the  following 
items: 

•course  number 

•guarter  taught 

•number- of  students  predicted 

•number  of  students  attending 
The  eventual  goal  of   the  analysis   became  that   of  finding 
such  permanencies  in  the  data  (based   on  these  items)   as  to 
provide  an  accurate  prediction  adjustment. 

1  •   Prediction  Data 

Prediction  data  was  available   both  for  individual  curricula 
inputs   into   a  given  course   and   for  the   total   expected 
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student  demand  for  those  courses.  This  data  was  presented 
in  two  formats:  summary  data  and  individual  course  data. 
The  summary  data  (figure  2. 1)  existed  for  academic  years 
1982,  1983  and  the  present.  This  data  provided  a  given 
years  display  of  the  total  expected  student  loading  indexed 
by  course  and  quarter.  It  included  a  recommended  number  of 
"sections  taught"  and  the  corresponding  faculty  hours 
required.  As  seen  in  the  figure,  the  predictions  are  made 
for  a  given  academic  year,  in  this  case  84  (September  1983 
to  August  1984)  .  The  first  column  contains  the  alpha- 
numeric course  identifier  as  listed  in  [Ref.  2].  The  'STCT' 
columns  refer  to  the  number  of  students  predicted  to  attend, 
the  •SEC1  column  refers  to  the  number  of  sections  required 
to  accomodate  those  students  (maximum  section  size  is 
limited  to  30  attendees),  and  the  'FAC-HRS'  columns  refer  to 
the  number  of  faculty  teaching  hours  needed  to  teach  those 
sections. 
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For  1983  and  1984,  individual  course  predictions 
were  available  (figure  2.2).  These  course  predictions  listed 
the  expected  student  loading  for  a  given  course  during  a 
given  guarter  coded  by  degree  awarded.  This  degree  coding 
is  found  in  the  "S/D/T"  (specialty/degree/time)  column  of 
the  data.  The  first  two  letters  of  this  column  apply  to  the 
curriculum  specialty,  the  next  two  to  the  degree  (MS  refers 
to  Master  of  Science).  This  coding  is  easily  tr ansf errable 
to  curriculum  number  using  Naval  Postgraduate  School 
instruction  1520.19  (figure  2.3).  The  numbers  that  occa- 
sionally appear  following  those  four  letters  indicate  when 
curricular  officers  have  not  provided  an  updated  list  of 
whom  they  expect  for  that  course  at  that  time  in  that 
specialty.  The  numbers  mean  that  the  prediction  listed  came 
from  the  "standard  files"  of  the  programming  office;  a 
collection  of  best  guesses  of  the  expected  number  of  atten- 
dees loosely  based  on  historical  patterns.*  These  individual 
course  predictions,  however,  make  no  mention  of  segment 
assignments. 


♦Cther  columns  in  the  individual  course  prediction  data 
are  generally  self-explanatory  with  the  exception  of  these 
minor  items:  The  AV  column  referred  to  the  number  of  avia- 
tors (Pilots,  NFOs,  etc.)  that  attend  and  is  an  unused  relic 
of  some  past  program.  The  letters  sometimes  preceding  the 
section  column  are  now  meaningless.  The  section  column  code 
itself  is  assigned  by  the  curricular  offices  to  groups  of 
incoming  students. 
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NAVPGSCOLINST    1520.19    CH-1 
21    October    1983 


CODES    USED    FOR    SCHEDULING 

A5~0F 

JUNE    1983 

SPECIALTY 
DEGREE 

CODE    30      CODE  SPECIALTY 

360  RAMS  OR/SA    ARMY 

RGMS  OR/SA    GENERAL 

RNMS  OR/SA    NAVY 

RSMS  OR/SA    SUPPLY    CORPS 

RPPH  OR/SA    PHD    IN    OR 

RCMS  GRCCP    OF    ANY    OF    ABOVE 

CODE    31 

€10      ACMS  AERONAUTICAL  ENGINEERING 

611      AXMS  AERONAUTICAL  ENGINEERING  (Avionics) 

ATMS  AERC  ENGINEERING  FOR  TEST  PILOTS 

CODE  32 

600  DCMS  ELECTRICAL    ENGINEERING    (Communications) 

590  EAMS  ELECTRICAL  ENGINEERING 

620      HCMS  COMMUNICATION  MANAGEMENT  (Coast  Guard) 

HMMS  COMMUNICATION  MANAGEMENT 

595      KEMS  ELECTRONIC  WARFARE  SYSTEMS 

366      SOMS  SPACE  SYSTEMS  OPERATIONS 

591  SEMS  SPACE  SYSTEMS  ENGINEERING 
CODE  33 

380  GMMS  MATH  OPTION  OF  ADVANCED  SCIENCE 

381  GPMS  PHYSICS  OPTION  OF  ADVANCED  SCIENCE 

530  WTMS  WEAPONS  SYSTEM  ENGINEERING 

531  WSMS  WEAPONS  SYSTEM  SCIENCE 

532  WNMS  NUCLEAR  PHYSICS  (Weapons  and  Effects) 
535      UXMS  UNDERWATER  ACOUSTICS 

CODE  331 

525      IXMS  ANTISUBMARINE  WARFARE 


Figure  2.3    Partial  Listing  of  Degree  to  Curriculum  Codes 
(see  [Ref.  3]  or  Appendix  B  for  complete  listing). 
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2 .   Historical  Data 

Historical  attendance  data  was  made  available  by  the 
registrar's  office.  This  data,  filed  by  quarter5  listed  the 
courses  taught  in  alpha-numeric  order  displaying  student 
names,  curricula,  and  grades  received  (see  figure  2.4).  The 
numbers  recorded  reflect  the  number  of  students  completing 
the  course  vice  the  number  seated  on  the  first  day  of  class. 
Segements  are  listed  separatly.  Use  of  this  historical  data 
implies  a  number  of  actions.  First,  the  predictions  are 
demand  based  and  would  probably  correlate  better  with  the 
number  of  students  registered  for  the  course  than  with  the 
number  completing.  A  strong  positive  correlation  is  assumed 
between  number  of  students  registered  and  number  completing 
a  given  course.  Second,  grade  and  name  information  was 
discarded  thereby  burying  any  measure  of  correlation  between 
performance  and  attrition.  Third,  segment  data  was  collapsed 
to  match  individual  course  prediction  data.  This  obscured 
relationships  betw-een  predicted  and  actual  number  of 
sections  taught  or  faculty  hours  required.  Since  the  goal  of 
the  analysis  was  to  adjust  the  predictions  of  student  demand 
vice  to  explain  the  reasons  an  adjustment  is  necessary, 
these  actions  were  ccosidered  acceptable. 


5The  registrar's  system  is  rather  unique  in  its  coding 
of  this  data.  'School  Year  84'  does  not  refer  to  fiscal 
year  84  (normally  1  October  1983  -  3TJ~September  1984)  . 
Instead.  'School  Year  84'  refers  to  the  September  84  to 
August  85  quarters.  As  this  data  was  collected,  the  coding 
of  quarterly  data  [to  be  discussed  later)  was  translated 
bacK  intc  standard  fiscal  year  notation.  Thus,  throughout 
the  data  base  used  in  the  analysis,  83.1  refers  to  the 
September  82  -  December  82  quarter  of  the  1982  -  1983  school 
year,  as  expected. 
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■  — 

OFFICIAL  GRADE  ROSTER 

PAGE  1 

NAVAI 

POSTGRADUATE  SCHOOL 

MONTEREY, CALIFORNIA   93943 

DATE  13  APR 

COURSE  NO.  ( 

3S3602  SEGEMENT 

00 QUARTER 

1   AY  83-84 

COMEAT 

MODELS 

PROF. 

SHUDDE,  REX  H. 

3  LECTURE 

HCURS   1  LABORATORY 

HOURS 

CORPS/ 

NAME 

RANK 

COUNTRY 

CURR 

MARK 

1. 

xxxxxx, 

xxxx 

X 

LT 

USN 

#374 

A 

2. 

xxxxxx. 

xxxx 

X 

LCDR 

USN 

#374 

B 

3. 

xxxxxx, 

xxxx 

X 

LCDR 

USN 

#525 

B 

4. 

xxxxxx, 

xxxx 

X 

LT 

USN 

#525 

B 

5. 

xxxxxx, 

xxxx 

X 

LCDR 

USN 

#525 

A- 

6. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

A 

7. 

xxxxxx, 

xxxx 

X 

LCDR 

USN 

#374 

A- 

8- 

xxxxxx, 

xxxx 

X 

LT 

USN 

#525 

A 

9. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

10. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

1  1. 

xxxxxx, 

xxxx 

X 

LCDR 

USN 

#525 

B 

12. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#525 

B 

FOE* 

i  4 

THIS  IS  THE 

OFFICIAL 

GRADE 

ROSTER  TO 

BE  FILED  IN  THE 

ACAEEMIC  DEPARTMENT 

Figure  2.4    Sample  Attendance  Data  (names  removed) 

3 .   Coding  System 

Before  entering  the  data  into  computer  files,  a 
coding  system  was  devised  to  distill  the  raw  data  into  its 
usable  parts.  Excess  information  such  as  student  names  and 
grades  could  then  be  discarded.  The  data  consisted  of 
predictions  and  actual  attendance  figures  coupled  with  iden- 
tifiers. Identifiers  required  include  course  number  and 
quarter  taught.  Available  data  included  both  total  and  indi- 
vidual curricula  figures.  For  simplicity  of  use,  a  basic 
file  was  created  using  the  following  five-column  coding 
scheme: 
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Quarter  Curricula*  Course*  #attended  #predicted 

Once  these  items  had  been  tabulated,  an  easy  subtraction 
(attending  minus  predicted)  provided  an  error  term.  By 
ordering  (by  magnititude  of  error)  and  displaying  the 
resulting  array,  a  history  of  many  poor  predictions  was 
observed.  This  observation  confirmed  that  a  problem  existed 
and  showed  a  need  for  further  analysis. 

B.   NATOBAI  DIVISIONS  IN  CODED  DATA 

The  large  volume  of  coded  data  created  was  divisible 
into  several  sub-categories.  The  Operations  Research 
Department  teaches  twc  types  of  courses:  OA  -  taken  by  oper- 
ations analysis  major  students;  and  OS  -  taken  by  students 
from  ether  curricula.  Initially  both  of  these  types  were 
taken  under  consideration  for  analysis.  Later,  the  OA 
designated  courses  were  dropped  from  consideration  for  two 
reasons;  the  problem  of  not  knowing  how  many  students  were 
going  to  show  up  was  generally  solvable  within  the  depart- 
ment; additionally,  time  constraints  limited  the  scope  of 
the  areas  to  be  covered.6  The  OS  course  data  was  broken  into 
two  prediction  areas:  summary  predictions  and  curricula 
specific  predictions.  The  summary  predictions  were  used  to 
show  patterns  by  course;  the  curricula  predictions  to  show 
patterns  by  individual  curricular  offices. 

1 .   Summary  Data 

When  dealing  with  summary  prediction  data  error 
terms,  the  goal  of  the  analysis  was  to  provide  a  functional 
relationship  between   predictions  and  attendance   by  course. 


6CA  data  collected  during  the  initial  stages  of  problem 

investigation   is    available.  The   format   is   slightlv 

different   from   that  of   the   OS  course  data   because   the 

problem  found  for  OA  dealt  with  elective  selection.  The 
factcr  of  interest  was  service  affiliation.  For  further 
details,  see  appendix  A. 
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Using  these  derived  relationships  (hopefully  simple  addi- 
tions or  subtractions),  the  department  chairman  will  be  able 
to  take  the  forecast  from  the  programming  office  (figure 
2.1)  and  change  the  figures  shown  to  reflect  more  accurately 
the  expected  student  load.  These  more  accurate  figures  can 
then  be  used  to  schedule  faculty  teaching  assignments, 
reguest  necessary  funding,  and  accomplish  other  tasks 
requiring  the  number  of  expected  students  per  course. 

2-   Curricula  Sp_ecific  Data 

Ehen  dealing  with  error  terms  for  individual 
curricula,  the  goal  of  the  analysis  was  to  provide  a  func- 
tional relationship  between  predictions  and  attendance  by 
curriculum.  Using  these  derived  relationships,  the  depart- 
ment chairman  will  be  able  to  take  the  forecast  for  an  indi- 
vidual course  (figure  2.2)  and  change  the  figures  shewn  to 
reflect  more  accurately  the  expected  student  load  by  either 
applying  the  relationship  or  calling  the  curricular  office 
involved  to  get  better  numbers.  These  historical  observa- 
tions will  enable  the  department  chairman  both  to  identify 
curricula  that  are  inadvertently  causing  scheduling  problems 
and  to  show  improvements  or  deterioration  in  the  scheduling 
process  itself.  This  information  can  be  used  both  by  the 
programming  office  and  the  curricular  office  involved  as 
well  as  the  OR  Department. 
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III.  DETAILED  ANALYSIS  OF  OS  PREDICTIVE  DATA 

At  the  completion  of  the  initial  steps  toward  analysis 
previously  discussed,  a  coded  series  of  data  arrays  were 
available  for  use.  An  example  of  array  formatting  is  shown 
in  figure  3.1  The  figure  of  interest  was  the  "error"  term 
found  in  column  six.  These  error  terms,  representing  the 
difference  between  the  actual  number  of  students  attending  a 
course  vs  the  number  predicted,  were  displayed  in  various 
groups  ordered  by  magnitude  to  show  the  accuracy  patterns  of 
the  forecast.  Areas  that  were  explored  for  effects  included 
the  curriculum  that  provided  the  input,  the  courses  them- 
selves, and  the  guarter  for  which  the  courses  were 
predicted.  Underlying  patterns  such  as  an  individuals 
"path"  through  various  OS  courses  required  for  his  curric- 
ulum were  also  investigated.  The  paucity  of  the  data 
precluded  the  use  of  sophisticated  statistical  techniques 
thereby  limiting  results  to  indications  of  areas  of  concern 
rather  than  definitive  statistical  statements. 

A.   TBE  DATA 

Six  element  data  arrays  (as  shown  in  figure  3.  1)  were 
created  for  each  years  data.  These  APL  variables  were 
combined  to  form  a  single  compilation  of  all  the  available 
data  (see  table  II) .  The  error  terms  of  this  compilation 
were  used  for  the  analysis.  These  error  terms  are  discrete 
integer  data  the  size  of  which  are  limited  by  the  maximum  of 
the  actual  or  predicted  columns  terms.  Distributional 
fitting  was  not  done  for  two  reasons;  the  population  should 
have  high  internal  correlation  due  to  the  interrelation 
between  course  errors  caused  by  student  course  matrices,  and 
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.   _. 

QTR 

CUERIC 

csz 

ACT 

PRE 

(ACT-PRE) 

**** 

Jjfc    ^S   Jp.    If.      J^fc     Jp. 

*** 

*** 

*** 

#*#  ***  *  ** 

8  4.3 

0 

2103 

0 

1 

-1 

84.3 

535 

2103 

0 

1 

-1 

84.3 

0 

3001 

25 

0 

25 

84.  3 

368 

3001 

25 

0 

25 

84.3 

0 

3002 

15 

18 

-3 

84.3 

683 

3002 

0 

1 

-1 

84.3 

825 

3002 

15 

17 

-2 

• 

83.1 

0 

3602 

7 

7 

0 

83.  1 

374 

3602 

1 

1 

0 

83.1 

525 

3602 

6 

6 

0 

83.  1 

0 

3603 

20 

21 

-1 

83.1 

595 

3603 

20 

21 

-1 

83.  1 

0 

3604 

18 

16 

2 

83.1 

365 

3604 

1 

0 

1 

QTR:  Quarter  taught.   83. 1  refers  to  the  first  guarter 
in  the  Sept.  82  to  Aug.  83  school  year. 

Curric:  Curricula  -  0  implys  summary  data  for  a  given 
course. 

Cse:  Course  number  (all  OS  courses) . 

Act:  Actual  number  of  students  listed  on  final  grade 
records  as  completing  course. 

Pre:  Predicted  number  of  students  (from  forecast) . 

Act-Pre:  Prediction  error  term. 


Figure  3.1    Example  of  Array  Format  (with  definitions) 

API  variable  used  :  OS. 


the  sample  sizes  of  those  homogeneous  populations  distilled 
from  the  data  (such  as  "all  365  students  taking  OS  3604  in 
the  second  guarter")  were  prohibitively  small.  Two  types  of 
errors  were  discovered,  predictive  errors  and  accuracy 
errors.  A  predictive  error  was  defined  as  either  the  fore- 
cast predicting  no  students  and  students  attending  or  vice 
versa.  Embedded  in  this  error  type  are  those  courses  which 
were  not   even  listed  but   taught  and  those   courses  dropped 
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TABLE  II 

API  Variables  used  in  Analysis 
(see  Appendix  A  for  details  or  use) 

VAEIAELE   SHAPE   DESCRIPTION 
********  *******  ************ 

OS82     69x6    AY  (Academic  year)  81-82  data; 

includes  summary  information  only 
(no  data  for  individual  curricula) 

AY  82-83  data;  complete  listing 

AY  82-83  non-entry  course  data  * 

AY  83-84  data;  only  gtrs  1,2  and  3 

Combination  of  all  of  above 
(without  repetition) 

*  see  'Course/Curriculum  Interactions  and  Tests' 
section  for  complete  description. 


OS83 

255x6 

OSTRK83 

140x8 

OS84 

228x6 

OS 

552x6 

from  the  schedule  in  the  budgeting  process.  The  same  divi- 
sion was  found  by  individual  curricula  (for  example,  curric- 
ulum 460  (engineering  science  -  preparatory)  is  never 
predicted  in  any  individual  course  forecast) .  Predictive 
errors  exist  when  cclumns  five  or  six  of  the  data  array 
contain  a  zero.  Accuracy  errors  are  other  differences 
between  the  actual  and  predicted  number  of  students. 

E.   CCOBSE/CURRICULUH  INTERACTIONS  AND  TESTS 

Each  individual  student  at  the  Naval  Postgraduate  School 
follows  an  assigned  course  matrix.  This  assignment  varies 
by  both  the  program  assigned  and  individual  student  needs. 
However,  all  matrices  list  a  series  of  courses  that  are 
either  reguired  or  elecive.  The  Operations  Analysis  curric- 
ulum, through  its  OS  course  offerings,  provides  a  service  to 
other  programs   by  providing   basic  and   advanced  pobability 
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and  statistics  training  not  available  in  other  curricula. 
These  service  courses  are  generally  required  by  ether 
curricula  as  a  prerequisite  to  both  further  training  and 
degree  completion.  As  such,  they  are  taught  in  a  certain 
sequence.  The  amount  of  training  required  differs  with 
different  programs,  so  various  OS  "tracks"  are  required. 
Sequencing  of  courses  in  this  manner  will  cause  dependence 
between  the  error  terms  of  certain  courses.  To  determine 
the  dependancy  relationship  within  the  data,  the  course 
matrices  for  all  curricula  offered  at  the  school  were 
collected  and  the  OS  course  sequences  extracted.  These 
sequences  were  combined  to  form  a  series  of  condensed  course 
matrices  as  applicable  to  the  Operations . Research  Department 
(see  figure  3.2  -  Note:  x's  refer  to  students  onboard  but 
not  taking  OS  or  OA  designated  courses) .  From  these 
matrices,  patterns  emerged.  As  with  any  program,  students 
started  with  basic  courses  and  moved  to  more  advanced 
topics.  Five  basic  courses  were  defined  as  "entry  level" 
courses.  One  of  these  courses  was  required  before  any 
advanced  work  was  done.  Since  the  entry  level  courses  were 
generally  taken  early  in  a  student's  time  in  his  program, 
and  since  the  entry  course  was  the  first  time  he  appeared 
eligible  for  "prediction",  the  entry  level  courses  were 
expected  to  have  the  highest  variance  in  error  terms.  This 
proved  to  be  true.  As  the  students  spend  more  time  in  their 
programs,  attrition  and  schedule  changing  should  level  off. 
Additionally,  since  most  students  spend  more  than  a  single 
year  going  through  their  programs,  "down-line"  predictions 
will  be  improved  merely  by  the  students  moving  from  the  "not 
yet  onboard"  to  the  more  stable  "attending  NPS"  prediciton 
group. 
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Definition  of  entry  level  courses  led  to  the  definition 
of  course  sequence  "tracks".  A  student  from  a  given 
curricula  will  take  an  "entry"  course  -  the  first  time  he  is 
seen  by  the  OA  Department  -  and  will  then  take  follow-on 
courses  hased  on  his  course  matrix.   Each  curriculum  has  its 


1 

! 

ENTRY 

CCDESE 
****** 

CURRICULUM 
********** 

TRACK  (Follow-on) 

COURSES,  ALL  OS 
***************** 

OS2103 

365 

3604,3404,3008, 
3603,4602 

366 

3604,3404,3008, 
3603,4602 

374 

3601,3602,3603, 
3604 

525 

3303,3604,3601, 
3401,4601,3602 

535 

none 

595 

3604,3003,3603, 
3403,4601 

OS3101 

367 

3004 

620 

3005 

686 

none 

687 

none 

825 

3404,3002 

Figure  3- 3   Example  of  Entry  vs  Track 
Course  Organization. 


own  "track"  (See  figure  3.3).  Using  these  tracks,  it  should 
have  been  possible  to  derive  deterministically  the  exact 
course  loading  for  those  follow  on  or  track  courses  given 
the  course  matrixes  and  attendance  figures  for  the  entry 
level  courses.  Using  the  FY83  data,  this  was  done.  The 
resulting  matrix  was  set  up  in  the  following  manner: 
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Quarter  Curric  Cse  Act  Pre  Dpre  (Act-Pre)  (Act-Dpre) 

Act   :  Actual  number  attending 

Pre   :  Standard  predictions 

Dpre  :  Eerived  predictions 

A  display  of  the  results  showed  no  observable  difference 
between  the  regular  error  term  and  the  derived  error  term. 
A  Wilcoxian  signed  rank  test7  was  done  to  test  the 
hypothesis  that  the  error  magnitude  of  the  adjusted 
predictions  tends  tc  be  smaller  than  the  magnitude  of  the 
error  frcm  the  forecast  (see  table  III) . 


TABLE  III 
Wilcoxian  Signed  Bank  Test  Results 

Sensitive  assumption:   Differences  independent. 
Assumed  large  amounts  of  varied  data  negated 
track  effects  resulting  in  relatively  independent 
difference  terms. 

CASE  A  CASE  B 

Variatles  Tested:   X  =Curric  ACT-DPRE   Sum  ACT-DPRE 

Y  =Curric  ACT-PRE    Sum  ACT-PRE 

H   :  The  X  values  do  not  tend  to  be  smaller  than 

0  the  Y  values. 

H   :  The  X  values  tend  to  be  smaller  than 

1  the  Y  values. 

Decision  Rule  :  Reject  at  the  5%  confidence 

level   if    TS    >    1.65 

CASE    A         CASE    B 
Test    Statistic    :  .453  -1.455 

Conclusion    :    Cannot   reject    null    hypothesis. 


7Taken   from    [Ref.    5    p.    280]. 
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The  results  of  the  test  indicated  that  the  adjusted 
predictions  were  nc  improvement  over  the  forecast.  That 
conclusicn  indicates  that  interactions  more  complex  than  the 
following  of  course  matrixes  are  involved  in  actual  student 
movement  from  course  to  course.  The  adjusted  prediction 
method  was  dropped  from  the  analysis  as  a  means  of  better 
predicting  course  loading. 


C.   CTEEB  INTERACTION  AND  TESTS 

Interactions  exist  in  areas  separate  from  the  dependen- 
cies created  within  a  class  sequence  by  the  course  matrix 
structure.  Three  main  effects  were  studied.  First, 
curricula  may  have  individual  patterns  across  time.  Some 
groups  may  consistently  have  an  optimistic  input  that  causes 
an  overestimation  of  number  of  students  wherever  those 
groups  appear.  Second,  individual  courses  may  show  patterns 
of  over-  or  underestimation.  Lastly,  a  quarterly  effect  on 
errors  may  appear  due  strictly  to  the  elapsed  time  between 
the  prediction  and  the  actual  class.  To  provide  an  initial 
look  at  the  data  to  show  such  patterns,  two  programs  were 
written.8  These  programs  were  designed  to  sort  the  data  into 
the  catagories  of  interest,  specifically  curriculum,  course, 
and  quarter.  The  resulting  groupings  showed  that  there  were 
patterns  in  each  catagory,  but  that  the  number  of  observa- 
tions available  to  base  conclusions  upon  made  those  patterns 
statistically  insignificant.  Through  clever  data  selection, 
a  series  of  observations  was  gleaned  to  test  for  curriculum 
and  course  effects  using  median  polish  technique  as  detailed 
in  [Eef.  4].     The  results  of   that  test   were  inconclusive 


8These  programs  and  others  used  in  the  analysis  are 
fully  explained  and  documented  in  appendix  A-  That  appendix 
also  contains  instructions  for  the  maintenance  and  use  of 
the  data  base  created,  and  procedures  for  gaining  access  to 
same. 

33 


showing  no  temporal  stability  (see  table  IV)  .  The  test  for 
quarterly  effects  was  not  done  due  to  the  paucity  of  the 
data. 9 


^Complete  data  existed  only  for  FY83.  This  limited 
sample  was  evaluated  as  too  unrepresentative  to  be  statisti- 
cally significant  concerning  quarterly  effects  across 
various  years. 
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IV.  CONCLUSIONS 

The  problem  as  defined  was  to  aid  the  Department 
Chairman  in  his  task  of  scheduling  and  budgeting  for  faculty 
teaching  responsibilities  by  providing  either  a  better 
prediction  method  or  adjustments  to  the  present  forecasts. 
Irom  the  schedulers  standpoint,  jumps  across  any  multiple 
of  thirty  students  cause  new  sections  to  be  created  or 
dropped,  and  courses  with  a  demand  of  less  than  five 
students  are  dropped.  To  the  scheduler  or  budget  office, 
these  are  significant  numbers.  The  Department  Chairman, 
however,  cares  little  about  the  actual  number  but  much  about 
the  accuracy  of  the  prediction.  Since  predictions  are  made 
by  course  and  broken  down  within  individual  courses  by 
curriculum,  adjustments  were  found  for  courses  and 
curricula.  Prediction  data  were  the  limiting  quantities; 
predictions  by  curricula  exist  only  as  far  back  as  FY83. 
This  limited  the  scope  of  the  investigation  to  1  and  3/4 
years.  With  this  limited  history,  indications  of  problem  or 
non-problem  areas  are  the  only  adjustment/accuracy  measures 
possible. 

A.   CCHCIOSIONS  BY  COURSE 

1  •   Program  0ut2U t  Dtili za tion  and  Displays 

Indications  of  predictive  accuracy  by  course  were 
obtained  using  the  * CCURSEP*  and  'COURSE1  programs  detailed 
in  Appendix  A.  The  'COORSEP1  (standing  for  course  patterns) 
program  was  used  first.  This  program  takes  the  input 
matrix,  in  this  case  the  variable  'OS'  containing  all  of  the 
data  so  far  available,  and  displays  it  grouped  by  course  in 
decreasing  order   of  predictive  error  magnitude   (see  figure 
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Figure  4.1    Sample  Output  from  COORSEP  Program 

4.1  for  a  single  course  example).10  This  output  is  quickly 
scanned  for  'interesting*  courses  -  those  with  either 
significant  problems  or  very  small,  infrequent  errors. 
Those  courses  with  problems  are  further  checked  to  determine 
if  there  is  any  pattern  to  the  errors  observed.  Many 
negative  numbers  will  indicate  an  overestimation  problem 
(prediction  greater  than  actual  attending) ,  conversely  posi- 
tive numbers  indicate  underestimation.  Quarters  may  be 
split  indicating  large  errors  for  some,  small  for  others. 
Curricula  may  be  grouped  together  indicating  prediction 
performance  differences.  In  the  example  shown  (Figure  4.  1) , 
error  magnitude  is  small  and  signs  vary,  indicating  rela- 
tively minor  deviations  from  the  predicted  values.  Had 
these  errors   been  large,   the  following   observations  could 


10In  this  display  (and  all  others),  the  quarter  is  shown 
as  a  negative  number.  This  is  done  strictly  as  a  formatting 
aid.  In  APL.  a  space  is  added  to  each  column  in  the  output 
display  to  allow  negative  signs.  The  error  terms  occasion- 
ally had  no  negatives.  When  this  occurred,  the  output 
display  deleted  the  six  spaces  'saved'  for  the  sign.  This 
caused  an  irritating  misalignment  in  the  columns  of  the 
display.  The  easy  correction  was  to  ensure  a  negative 
existed  in  all  cases,  thus  the  quarter  coding. 
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have  beer  made: 

•  Third  quarter  errors  tend  to  be  higher  than 

first  quarter  errors. 

•  No  curricula  appears  to  do  any  better  or  worse 

than  any  other,  though  374  may  tend  to  over- 
estimate in  their  predictions. 

To  further  detail  predictive  behavior  within  a 
course,  the  'COURSE*  program  is  utilized.  This  program 
takes  the  same  matrix  previously  investigated  (in  this  case 
•OS')  and  also  displays  it  grouped  by  course,  but  with 
various  catagorical  presentations  made.  Output  may  be 
displayed  in  quarterly  or  curricula  subcategories  (through 
answering  program  prompts)  .  With  the  curricula  option 
selected,  the  output  compares  patterns  of  individual 
curricula  (see  figure  4.2).  Statistics  displayed  include 
the  mean  and  variance  of  the  error  terms  for  individuals 
curricula,  and  predictive,  accuracy  and  no  error  percentages 
also  by  curricula.  To  avoid  counting  data  twice,  summary 
data  entries  (column  2=0)  are  disregarded.  From  scan  of 
the  example  (figure  4.2) ,  the  following  conclusions  can  be 
drawn : 

•  Curriculum  525  provides  the  majority  of  the 

input  for  OS3602,  and  their  predictions 
are  excellent  indicating  no  problem  with 
the  course. 

•  Curriculum  374  may  show  a  pattern  of  over- 

estimation. 

•  Curriculum  686  (with  100%  prediction  errors) 

normally  will  not  provide  inputs  to  this 
course. 
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Figure  4.2    Sample  Output  from  COOESE  Program 
Options  :  Curricula  Grouping  with  Error  Percentages. 


The  curricula  option  of  the  'COUESE'  program  appears 
to  be  the  most  valuable  display  available.  The  information 
shown  is  directly  transf errable  to  the  prediction  data  as 
presented  (refer  back  to  figures  2.1  and  2.2). 

With  the  quarter  option  selected,  the  output 
combines  all  of  the  data  into  four  catagories  depending  on 
the  guarter  in  which  the  course  has  been  taught  (figure 
4.3).  The  same  statistics  are  displayed,  this  time  for  each 
quarter.  Again,  summary  data  entries  are  disregarded  to 
avoid  duplication  of  data.  From  a  scan  of  this  figure,  the 
following  conclusions  can  be  drawn: 
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Figure  4.3    Sample  Output  from  COURSE  Program 
Options  :  Quarter  Grouping  with  Error  Percentages. 


•  OS3602  has  only  been  taught  during  the  fall 

and  spring  guarters. 

•  Fall  quarter  predictions  tend  to  be  exact. 

•  Spring  quarter  predictions  are  worse,  but  not 

to  such  an  extent  as  to  cause  problems. 


2 «   Dis£la_x  Bas_€2<3  Conclusions  by_  Course 

Using  the  previously  discussed  displays,  the  courses 
were  separated  into  three  catagories  of  interest:  those 
with  indications  of  significant  prediction  accuracy 
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problems,   those  with  relatively  minor  problems  if  any,   and 
those  with  too   little  information  to  draw   conclusions  (see 


TABLE  V 
Course  Results 

Problem  Courses  'Good'  Courses 

2101  2102  2103 

3001  3008  3101 

3102  3103  3104 

3105  3404  3702 

Inadequate  Data 

3003   3005   3302 
3401   3403   4602 


3002 

3004 

3006 

3303 

3402 

3601 

3602 

3603 

3604 

4601 

4701 

table  V) .  Predictions  made  for  courses  can  be  checked 
against  this  listing  for  an  indication  of  their  expected 
performance.  Should  more  detailed  information  be  needed, 
Appendix  C  contains  the  results  based  on  summary  data  from 
81-82,  complete  82-83  data,  and  fall,  winter  and  spring 
quarters  from  the  83-84  school  year. 

E.   CONCLUSIONS  BY  CUBBICULA 

1  •   -EE°_2£am  Output  Utilization  and  Displays 

Indications  of  predictive  accuracy  by  curricula  were 
obtained  using  the  ^ORRICP*  and  ■CURRIC1  programs  detailed 
in  Appendix  A.  The  'CURRICP'  (standing  for  curricula 
patterns)  program  was  used  first.  This  program  takes  the 
input  matrix,  'OS'  as  before,  and  displays  it  grouped  by 
curricula  in  decreasing  order  of  predictive  error  magnitude 
(see  figure   4.4  for  a  single   curriculum  example).     This 
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Figure  4.4    Sanple  Output  from  CORRICP  Program 

output  is  quickly  scanned  for  'interesting*  courses  -  those 
with  -significant  picblems  or  small,  infrequent  errors. 
Those  curricula  with  significant  errors  are  further  checked 
for  discernable  patterns  such  as  regularly  occuring  over-  or 
underestimation,  or  guarterly  trends.  In  the  example  shown 
(figure  4.4) ,  problems  appear  due  to  consistent  overestima- 
tion.  However,  given  the  magnitude  of  the  errors,  these  may 
be  viewed  as  insignificant.  It  also  appears  that  fall  and 
winter  quarters  tend  to  have  fewer  or  smaller  prediction 
errors. 

lo  more  closely  investigate  patterns  shown,  the 
•CURRIC  program  is  utilized.  This  program  takes  the  same 
input  matrix  and  displays  it  further  grouped  by  quarterly  or 
course  catagories.  With  the  quarter  option  selected,  the 
output  compares  predictions  across  time  (see  figure  4.5). 
From  the  example  shown,  the  following  observations  can  be 
made: 

•  Overestimation  errors  do  increase  across 
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consecutive  quarters. 

•  Lower  level  courses  tend  to  have  larger  mag- 

nitude errors  than  more  advanced  courses. 

•  OS2103  and  OS3401  might  not  be  taken  by  cur- 

riculum 374  students  fall  quarter  (more 
data  required) 
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Figure  4.5    Sample  Output  from  CURRIC  Program 
Options  :  Quarter  Grouping  with  Error  Percentages. 
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With  the  course  option  selected,  inf orma tior  can  be 
gleaned  on  curriculum  374's  prediction  performance  fcr  any 
given  course  (see  figure  4.6).  Again  from  the  example 
shown,  conclusions  can  be  drawn: 

•  OS2101  and  OS3401  are  generally  not  taken  by 

students  from  curriculum  374 

•  Larger  errors  cccur  in  OS2103  and  OS3604  (as 

expected  from  the  course  matrix  -  refer  back 
to  figure  3.2) 

2 .   Display  Base d  Conclusions  by  Curriculum 

Using  the  previously  discussed  displays,  the 
curricula  were  separated  into  three  catagories  of  interest: 
those  with  indications  of  significant  prediction  accuracy 
problems,  those  with  relatively  minor  problems  if  any,  and 
those  with  too  little  information  to  draw  conclusions  (see 
table  VI) .  Predictions  made  for  courses  can  be  checked 
against  this  listing  for  an  indication  of  their  expected 
performance.  Should  more  detailed  informtion  be  needed. 
Appendix  C  contains  the  results  based  on  summary  data  from 
81-82,  complete  82-83  data,  and  fall,  winter  and  spring 
quarters  from  the  83-84  school  year. 
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Figure   4.6        Saaple  Output   from   CIJBEIC  Program 
Options   :    Course  Grouping   with    Error   Percentages. 
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TABLE  VI 
Curricula  Results 


Prctlem  Curricula 


360  368 
531  590 
827  837 


374  460  530 
817  819  825 
847  857 


•Good'  Curricula 

365  366  367  377  525 
595  600  620  813  8 14 
815  999 


Inadequate  data 

372  373  380  440  441 
532  535  570  591  610 
611  683  686  687 
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APPEND!!  A 
PROGRAHS  AND  DATA  BASE  USERS  MANUAL 

This  appendix  contains  the  detailed  information  required 
to  effectively  run  and  maintain  the  programs  and  data  base 
used  for  historical  validation  of  the  Naval  Postgraduate 
School  course  loading  forecasts.  The  purpose  of  this  series 
of  programs  is  to  conveniently  display  historical  course 
attendance  data  versus  predicted  attendance  data.  These 
displays  can  then  be  translated  into  simple  adjustments  to 
be  made  to  the  forecast,  providing  a  more  accurate  figure  to 
te  used  in  the  faculty  teaching  assignment  process. 

A.   ACCESS  PROCEDURES 

The  data  and  manipulative  programs  associated  are  held 
on  the  IBM  3033  mainframe  at  the  Naval  Postgraduate  School 
under  user  number  1502P.  The  are  contained  in  an  APL  work- 
space called  'OSDATA1.  Assuming  you  have  a  valid  user 
account  on  the  main  NES  computer  system,  this  workspace  can 
be  accessed  in  three  steps;  linking  to  user  1502P,  accessing 
his  disk,  and  copying  the  workspace  onto  your  own  disk. 
Once  you  are  'logged  on'  to  the  system,  the  command 

CP  link  1502p  195  195  rr 

will  cause  a  prompt  to  appear  asking  for  the  'read'  pass- 
word. Enter  fps3000'.  That  password  allows  a.ccess  to  user 
1502P's  disk.  To  get  the  workspace  transferred  to  your  cwn 
disk,  enter 

ace  195  b/a  followed  by 

copy  osdata  vsaflws  b  =  =  a 
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B-   EDN  FBEPARATION  INSTRUCTIONS 

To  effectively  run  the  programs   available,   data  must  exist 
in  a  six  column  numerical  matrix  format  coded  as  shown: 

Column   Sample 

Number   Entries   Comments 

1  83.1    This  column  corresponds  to  the  quarter 
84. U    from  which  the  data  is  gleaned.  f83.V 

corresponds  to  the  Fall  quarter  of  the 
1982-1983  school  year  (as  in  standard 
fiscal  year  notation).   Similarly, 
84.4  corresponds  to  the  summer  quarter 
of  the  1983-1984  school  year. 

2  0    This  column  holds  the  curriculum  code. 
374    *  0'  means  that  the  entries  to  follow 

are  total  number  of  students  predicted 
and  total  actually  enrolled  in  the 
listed  course  as  compared  to  '374* 
which  means  that  the  figures  to  follow 
apply  only  to  curriculum  374. 

3  2103    This  column  holds  the  OS  course  number 
3602 

4  29    This  column  holds  the  actual  number  of 

0    students  attending  the  course  as  de- 
scribed by  the  previous  columns.   If 
column  two  is  zero,  this  number  is  the 
total  number  of  students  that  attended 
and  should  correspond  to  the  total 
listed  on  the  grade  record  sheet  held 
by  the  registrar.   If  this  number 
itself  is  zero,  the  course  was  not 
taught  during  the  quarter  shown  in 
column  one. 


48 


5  0      This  column  holds  the  predicted  number 
15      of  students  that  should  attend  the 

course  described.   If  column  two  is 
zero,  this  number  is  the  total  number 
of  students  that  were  predicted  for 
the  ccurse  and  should  correspond  to 
the  totals  listed  on  both  the  summary 
and  individual  course  forecasts.   If 
this  number  itself  is  zero,  either  the 
course  was  predicted  to  be  empty  or 
the  ccurse  was  not  predicted. 

6  -1      This  column  holds  the  difference 

0      between  the  actual  number  of  students 
5      attending  the  course  and  the  number 
predicted  (actual  minus  predicted). 
Negative  numbers  imply  overestimates, 
positive  numbers  imply  underestimates. 

Prior  to  any  program  runs,  the  goal  of  your  analysis 
must  be  defined.  The  programs  are  generally  divided  into 
two  areas,  course  centered  and  curricula  centered.  In  the 
course  centered  area,  the  'COURSEF'  program  allows  a  general 
viewing  of  the  error  data  for  chosen  courses.  This  prcgram 
displays  the  data  separated  into  courses  ordered  by 
decreasing  magnitude  of  the  error  term  (actual  minus 
predicted;  column  six)  and  is  used  to  check  quickly  the 
prediction  accuracy  by  course.  Then  to  check  for  underlying 
patterns  by  guarter  taught  or  by  individual  curricula,  the 
'COURSE'  program  can  be  used.  Similarly,  in  the  curricula 
centered  area,  the  'CURRICP*  program  allows  a  general 
viewing  of  the  error  data  separated  into  individual 
curricula,  and  the  'CURRIC*  program  can  be  used  to  further 
distill  the  informaticn  available. 
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In  summary,  two  steps  must  be  taken  prior  to  starting 
program  runs: 

•  Ensure  the  data  exists  in  the  proper  format 

•  Decide  what  information  is  needed  (define  goals) 

C.   SAME1E  PEOGBAM  BOSS 

To  demonstrate  effectively  the  varied  uses  of  the 
programs  available,  a  hypothetical  problem  will  be  created 
and  fully  explored.  Each  program  will  be  discussed  in 
detail  as  it  appears  in  the  problem  solution,  and  both  the 
inputs  required  and  output  generated  will  be  shown. 

1  •   Problem  Statement 

Curriculum  525,  Antisubmarine  Warfare,  and  curric- 
ulum 374,  Air-Ocean  Tactical  Environmental  Support,  are 
going  to  combine  parts  of  their  programs  to  create  a  new 
curriculum:  Curriculum  449.5  -  Antiwhale  Warfare.  The 
condensed  course  matrix  for  curriculum  449.5  has  OS2103, 
CS3604,  OS3601,  and  OS3602  in  quarters  2,  3,  4  and  5  respec- 
tively. As  the  Operations  Analysis  Department  Chairman,  you 
know  that  OS2103  and  OS3604  can  handle  the  additional 
students  with  no  problems  due  to  the  present  size  and 
frequency  of  those  course  offerings.  Your  question,  with 
scheduling  coming  up  for  the  next  fiscal  year,  is  whether 
the  additional  12  students  you  expect  per  'class1  will  be 
enough  to  require  the  offering  of  an  additional  section  for 
OS360  1  or  OS3602.  The  figure  of  twelve  students  comes  from 
the  Air-Ccean  curricular  office,  since  curriculum  449.5  will 
te  taught  as  an  adjunct  of  curriculum  374.  These  students 
will  be  entering  in  the  Fall  and  spring  quarters. 
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2  •   Definition  of  Goals 

The  first  step  in  the  problem  solution  is  to  define 
the  information  required  to  answer  the  question.  In  this 
case,  we  need  to  know  if  an  additional  section  is  required 
given  an  influx  of  twelve  additional  students.  This  problem 
defines  three  questions: 

••  How  many  students  usually  take  the  course  in 
question 

•  How  accurate  are  those  figures 

•  How  good  is  the  estimate  of  twelve  additional 
students 

Since  the  first  items  to  be  considered  involve 
course  data,  the  course  related  programs  will  be  used.  To 
check  later  the  accuracy  of  the  figure  twelve,  a  curricular 
comparison  can  be  made  with  curriculum  374  (since  that 
office  is  the  administrator  of  the  program)  using  the 
curricula  related  programs.  Additionally,  the  data  used 
must  be  the  most  recent  and  complete  information  available. 

3 .   Course  Related  Work 

The  first  step  we  will  take  is  to  get  a  look  at  the 
data  of  interest.  The  variable  'OS'  contains  the  informa- 
tion we  want.  The  'COORSEP*  program  most  easily  shows  this 
data.  Through  proper  inputs,  we  can  call  up  the  historical 
performance  of  OS3601  and  OS3602.  The  'COURSEP'  program 
provides  all  the  prompts  necessary  (see  figure  A.1).  The 
program  output  is  shown  in  figure  A. 2  .  See  figure  A. 3  for 
a  program  listing. 

As  we  can  see  from  figure  A. 2,  the  actual  number  of 
students  that  attend  each  course  appears  to  be  somewhere 
around  15.  The  predictions  are  presently  perfectly  adequate 
since  they   show  a  requirement  for   only  one  section   in  all 
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cases,  which  holds  true.  With  the  addition  of  twelve 
students,  however,  the  number  of  students  will  be  close  to 
30,  necessitating  another  section.  To  more  adequately  show 
the  data,  we  can  lcok  at  the  'COURSE1  program  sorted  by 
curricula.  This  will  break  out  the  summary  predictions  to 
tetter  show  how  well  the  course  is  predicted.  Figure  A.  4 
shows  the  inputs,  figure  A. 5  the  outputs,  and  figure  A. 6  the 
program  itself. 

Using  only  the  summary  data  shown  in  figure  A. 5,  we 
can  confirm  that  the  actual  number  of  students  that  attend 
each  course  is  about  15,  and  the  present  one  section 
requirement  holds  true.  We  can  further  observe  a  possible 
pattern  ly  quarter  (errors  appear  to  increase  for  the  fourth 
quarter  predictions  as  compared  to  predictions  made  for 
earlier  quarters) .  To  more  adequately  show  this  pattern  in 
the  data,  we  can  look  at  the  'COURSE'  program  sorted  by 
quarter.  This  will  break  out  the  predictions  to  better  show 
quarterly  patterns.  Figure  A. 7  shows  the  inputs,  figure  A. 8 
the  outputs. 

In  summary,  through  use  of  the  course  related 
programs  we  have  found  that  a  course  size  of  approximately 
fifteen  is  usual  for  these  courses.  Predictions,  especially 
those  made  for  the  fourth  quarter,  tend  to  overestimate  the 
number  of  students  expected.  If  the  figure  of  twelve  addi- 
tional students  is  in  fact  correct,  a  new  section  will  prob- 
ably not  be  required  even  though  predicted  attendance  values 
are  in  the  twenties  without  the  addition  of  the  new 
curriculum. 
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INPUT  PHQMF1 

COORSEP    OS 

0 

3601    3602 


*  see  COORSEP  listing  for  details 
of  the  prompts 


Figure  A-1    Input  Syntax  :  COORSEP 
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COURSE   t 

3601 

"83*4 

0 

3601 

14 

23 

"9 

"83.4 

365 

3601 

3 

11 

"8 

'82*4 

0 

3601 

14 

21 

"7 

"82.2 

0 

3601 

14 

8 

6 

"83.4 

374 

3601 

2 

5 

"3 

"83.2 

0 

3601 

14 

16 

_2 

"83.4 

525 

3601 

9 

7 

2 

"84.2 

365 

3601 

2 

0 

n 

"84.2 

525 

3601 

20 

22 

"2 

"83.2 

374 

3601 

4 

5 

"1 

"83.2 

525 

3601 

10 

11 

"1 

"84.2 

0 

3601 

25 

26 

"1 

"84.2 

374 

3601 

3 

4 

"1 

COURSE  ♦ 

3602 

"82.1 

0 

3602 

"83.3 

374 

3602 

"84.3 

374 

3602 

"84.3 

525 

3602 

"82.3 

0 

3602 

"83.3 

0 

3602 

"83.3 

686 

3602 

"83.1 

0 

3602 

"83.1 

374 

3602 

"83.1 

525 

3602 

"83.3 

525 

3602 

"84.1 

0 

3602 

"84.1 

374 

3602 

"84.1 

525 

3602 

"84.3 

0 

3602 

7  11  "4 

3  5  "2 

2  4  "2 
12  10  2 
15  14  1 
15  16  "1 

10  1 

7  7  0 

1  1  0 

6  6  0 

11  11  0 

12  12  0 

3  3  0 
9  9  0 

14  14  0 


Figure  A. 2   Output  froa  Input  Sequence  shown  in 
Figure  A. 1  :  C00RSEP  Program. 
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<7     COURSEP     MATRl:<;:<jIjT|Z 

C13   n 

[23    A   THIS  FUHCTIOH,   GIVEN  UH  INPUT  MATRIX  IN  THE  FOLLOWING  FORM   • 

C33    A 

C4D    R         OTR    CURRIC    CSE    ACTUAL    PREDICTED    ( ACT U AL - PR E D I C TE D ) 

C53   a 

161  A   WILL   SORT  AMD  DISPLAY   THE   DATA  BY  CURRICULUM   IH  THE 

C73    A  FOLLOWING  MANNER   { 

L81       a 

C93    A        COURSE 

ClOD  a 

Cll]   s  DATA  ORDERED  BY  MAGNITUDE  OF  ERROR  TERM 

C123  a 

C13]   A   (ALL   GUARTERS  WILL   BE  DISPLAYED  AS   NEGATIVE  NUMBERS,   THIS  ALLOWS 
C143  A    FOR  CORRECT  PLACEMENT  OF  THE  OUTPUT  COLUMNS  WITHOUT  RESORTING 
C15D   A    TO  COMPLEX  CHARACTER  ARRAY  MANIPULATIONS.) 

ci6:  R 

{.171  A   THE  PROGRAM   GOAL   IS  TO  DISPLAY  OBVIOUS  PATTERNS  THAT  MAY   THEN   BE 

C183  A  LOOKED  AT  MORE  CLOSELY  USING  THE   'CURRIC"   OR   'COURSE'   FUNCTIONS, 

C193  A  COURSES  MAY  BE  SELECTED  INDIVIDUALLY   IF  SO  DESIRED, 

C203  A 

C213    MATRIX«-MATRIXX  (  f  MATRIX  )  f      "111111 

C223    t-'IP   f°U  WANT  ALL  COURSES  DISPLAYED,   ENTER   1' 

C233   r' otherwise,  enter  o  to  select  some  subset  of  the  courses' 

£24]  -»UUMPlxiQ 

C25D  t ' THE  FOLLOWING   COURSES  ARE  AVAILABLE  FOR   SELECTION   {■ 

C26D  tX«-^C*-«-UNIGUE  MATRIXCJ3]] 

Z111  r  ' ENTER   THE   COURSE  NUMBERS  YOU  WANT,   ENTER   X  TO  GET   THEM  ALL,' 

C283  T'(NOTE   •   YOU  MUST  CHOOSE  MORE   THAN  ONE  COURSE)' 

L291       T«-fx«-a 

C303   -»jump2 

C313  JUmpi  •  Y«-f  x<-xc iXfUNinuE  matrixl;333 

C323  JUMP2ti«-l 

1131      CRSLOOPtf'   ' 

C343  -»Ox\i>^ 

C35D  (r  'course  •  '  )  ,T><Ci] 

1261  z<-(«Ci]  =  »*»TRi;<C;33)/Ci:  matrix 

C373  fZ  order  £ 

C38]  r'      ' 

C393  IH  +  1 

C40:  *'   ' 

C413    -»CRSLOOP 
■7 


Figure  A. 3    'COaRSEF'  Program 
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INPUT  PROMPT 

CCURSE    OS 


3601    3602 


E 


*  see  COURSE  listing  for  details 
of  the  prompts 


Figure  A. 4    Input  Syntax  :  COURSE 
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- — 

COURSE   ; 

3601 

"83.4 

0 

3601 

14 

23 

"9 

"82.4 

0 

3601 

14 

21 

"7 

"82.2 

0 

3601 

14 

8 

6 

"83.2 

0 

3601 

14 

16 

—  2 

"84.2 

0 

3601 

25 

26 

"1 

"83.4 

365 

3601 

3 

11 

"8 

"84.2 

365 

3601 

i 

0 

n 

"83.4 

374 

3601 

2 

5 

"3 

"83.2 

374 

3601 

4 

5 

"1 

"84.2 

374 

3601 

3 

4 

"1 

"83.4 

525 

3601 

9 

7 

2 

"84.2 

525 

3601 

20 

22 

~2 

"83.2 

525 

3601 

10 

11 

"1 

GROUP 

MEAN 

VARIANCE 

N    (OF 

ERROR 

TERM) 

365.0 

"3.0 

50.0 

2 , 

0 

374.0 

"1.7 

1.3 

3, 

0 

525.0 

".3 

4.3 

3. 

0 

OVERALL : 

"1.5 

10.0 

8.0 

COURSE   : 

3602 

"82.1 

0 

3602 

7 

11 

"4 

"82.3 

0 

3602 

15 

14 

1 

"83.3 

0 

3602 

15 

16 

"1 

"83.1 

0 

3602 

7 

7 

0 

"84.1 

0 

3602 

12 

12 

0 

"84.3 

0 

3602 

14 

14 

0 

"83.3 

374 

3602 

3 

5 

- 2 

"84.3 

374 

3602 

2 

4 

~2 

"83.1 

374 

3602 

1 

1 

0 

"84.1 

374 

3602 

3 

3 

0 

"84.3 

525 

3602 

12 

10 

2 

"83.1 

525 

3602 

6 

6 

0 

"83.3 

525 

3602 

11 

11 

0 

"84.1 

525 

3602 

9 

9 

0 

"83.3 

686 

3602 

1 

0 

1 

GROUP 

MEAN 

VARIANCE 

N     (OF 

ERROR 

TERM) 

374.0 

"1.0 

1.3 

4 

.0 

525.0 

.5 

1.0 

4 

.0 

686.0 

1.0 

1.0 

1 

.0 

overall; 

~.l 

1.6 

9. 

0 

Figure  A. 5    Output  from  Input  Sequence  shown  in 
Figure  A. 4  :  COURSE  Program* 
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9COURSE[[]]7 

<7  COURSE   MftTRI!(;X(I)Y;ZiMCJVC;jJ!<2!T2rHa)VOiW)OTR;na(HC)MS)NEiIHDiflE;PE 

CI  3 

ft 

C2: 

ft 

THIS  FUNCTION,   GIVEN   AH   INPUT   MATRIX   IN  THE  FOLLOWING  FORM   • 

C33 

ft 

C43 

ft 

QTR    CURRIC    CSE    ACTUAL    PREDICTED    (ACTUAL-PREDICTED) 

C33 

ft 

C&3 

ft 

WILL  SORT  AND  DISPLAY  THE  DATA  BY  CURRICULUM   IN  THE 

C73 

ft 

FOLLOWING   MANNER   . 

C83 

ft 

C93 

ft 

COURSE 

C103 

ft 

QUARTER   J,   DATA          CURRICULUM  \      DATA 

en: 

ft 

QUARTER  2   DATA    OR    CURRICULUM  2  DATA 

C123 

ft 

QUARTER   3   DATA                     , 

C133 

ft 

QUARTER  4  DATA          CURRICULUM  N  DATA 

C143 

ft 

MATRIX  OF  GROUP  MEANS,   VARIANCES  AND  N 

C153 

ft 

OVERALL   MEAN,   VARIANCE   AND   H 

C163 

ft 

C173 

ft 

A  MATRIX  OF  ACCURACY,   PREDICTIVE  AND  NO  ERROR  PERCENTAGES  IS 

C183 

ft 

AVAILABLE  IF  DESIRED, 

C193 

ft 

C2o: 

ft 

(ALL  QUARTERS  WILL  BE  DISPLAYED  AS  NEGATIVE  NUMBERS,   THIS  ALLOWS 

C213 

A 

FOR   CORRECT  PLACEMENT   OF   THE  OUTPUT  COLUMNS  WITHOUT  RESORTING 

C223 

ft 

TO  COMPLEX  CHARACTER  ARRAY   MANIPULATIONS.) 

C233 

ft 

C243 

ft 

XXXXXXXX   NOTE   ;   THE   MEAHS   AMD   VARIANCES   COMPUTED   APPLY     >|>ll<t> 

C253 

ft 

XXXXXXXX              ONLY   TO  THE   INDIVIDUAL  CURRICULQ   I  i  i  i  i     XXXXXXXX 

C263 

ft 

XXXXXXXX              AHY  SUMMARY   DATA   (^COLUMN  2  EHTRY  =  0)     XXXXXXXX 

C273 

ft 

XXXXKXXX             IS  COMPRESSED  OUT  OF  THE  COMPUTATION.      XXXXXXXX 

C283 

ft 

XXXXXXXX              THIS  CAN  BE  CHANGED   BY   REPLACING  THE   *    XXXXXXXX 

C293 

ft 

XXXXXXXX             SIGN   IN  THE  LOOPS   (WHERE  =  OR  W  CJ23   x*  XXXXXXXX 

C303 

ft 

XXXXXXXX               COMPRESSING   COLUMN   £   OR   WHERE  .<2      IS          XXXXXXXX 

C31U 

ft 

XXXXXXXX               COMPRESSING   THE   OUTPUT)   WITH   '=',   THISXXXXXXXX 

C323 

ft 

XXXXXXXX               FUNCTION   WILL   THEN   DISPLAY   ONLY   THE          XXXXXXXX 

C333 

ft 

xxxxxxxx           SySiSSSI  DATA,                                   XXXXXXXX 

C343 

ft 

C353 

ft 

COURSES  MAY  BE  SELECTED  INDIVIDUALLY   IF  SO  DESIRED, 

C363 

ft 

C371 

MATRIX«-MATRIXX  (  ?  MATRIX  )  f      "111111 

C383 

r'IF   i'OU  WANT  ALL  COURSES  DISPLAYED,   ENTER   1' 

C393 

t  I  OTHERWISE ,   ENTER   0   TO  SELECT   SOME   SUBSET  OF  THE  COURSES' 

C403 

-»JUMPlx\D 

C413 

r'THE  FOLLOWING  COURSES  ARE  AVAILABLE  FOR  SELECTION   •< 

C423 

r^<.X[4XfUNIQUE  MATRIX[(3]] 

C433 

T'ENTER   THE  COURSE   NUMBERS   YOU  WANT,   ENTER   X   TO  GET  THEM  ALL,' 

C443 

r'(NOTE   :   YOU  MUST  CHOOSE  MORE  THAN  ONE  COURSE)' 

C453 

r«-rx«-a 

C46: 

-»JUMP2 

C473 

jumpi;y*./>x«-xl±x«-unique  matrixc»33  3 

C483 

jump2  S  I«-l 

C493 

t'THE   DATA  MAY   BE  DISPLAYED   (TO  LOOK   AT  PATTERNS)   IN  GROUPINGS   BY' 

C503 

t'EITHER  QUARTER  OR  CURRIC,    FOR  QUARTER  GROUPINGS,   ENTER   1   ;   FOR' 

C513 

r'CURRIC  GROUPINGS,   ENTER  0' 

C523 

v«-a 

C533 

^'ACCURACY  ERROR,   PREDICTIVE  ERROR,   AND  NO  ERROR  PERCENTAGES  ARE' 

C54: 

^.'AVAILABLE,   ENTER   Y   IF  DESIRED,  H      OTHERWISE' 

C553 

IND«-Q 

C563 

CRSLOOP;r'   ' 

C573 

-►Ox  \  I  >  r 

C583 

(T'COURSE   J   '  )  PTi<CI] 

C591 

:Hxcn=HftTRi:<[;3])/[i]  matrix 

Figure  A. 6 
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C6o: 
C6i: 
C6: 

C63! 
C64! 

las: 

C6<s: 

C67: 

C68: 
C69: 

c7o: 
C7i: 

IT. 

c?z: 

C74! 
C7S 

C76: 
C77! 
C78I 

C7?: 
:8o: 
est: 

C82: 
C83! 
C84! 

ess: 

C86! 
C87: 

ess: 

C89: 

c9o: 
C9i: 

C92: 

:93: 

C94! 

C9: 


MC«.MEftN(Zc;23^0)/2Cf63 
vc«.vaRiA(Zi;;2:!<0)/2C;6: 
NC4-f  (zc;23^0)/2C}63 

J«-l 

-»ne:<tx  \  v^i 

oTft«-iox(-zc;i:)-L-zc;i3 

->OUT 
NEXT  ;  OTR(.Z[  ;  23 

out  •  Y2«-pX2«-x2[;4;<2*-uNiouE    otr] 

NSf«E(-PE».NOfMa(.VQ(.l() 
aTRLQOP;^EHDX)J>r2 

"«-(  (:<2C-i:  =  aTR)/ci3  =)  order  6 
n<a«.Ma,  mean  <wc;  23*0  )/»»C  !4] 
vQ«-vQ,vftRift(wc;23?!0)/«C?63 

NQ+.NO ,  NE«-f  ( wc ;  23  *o )  /»*c ;  6  3 

4JUMP3X  \  IND=  '  N  • 

pe«-pe,  (+/(»c;23»«0)/<»»C;4  3=0)vMC;53=0)-rHe 
ae*.ae,  (+/(  wc  ;  23*0  )/(wc;4  3^0)  a  ( «C » 53*0 >a"C» 63*0  )tNE 
ns«-ns,  <+/<wC}23*0)/("C  J4  3*0>a<'"C;5  3*0>awCS63=0>-t*e 
JUMP3 ; rw 

T  '   ' 
+OTRLOOP 

end;  i«-i  +  i 
r '   group     mean   variance   n   (of  error  term)' 
7  1  t(X2^0)/C13(  (S><2»C0.53  *a>fC23  va)»C23  no 

* ' 

(t'OVERftLLJ       '  )f      6     1     **C,VC,NC 

<■ '      ' 

-.JUMP  4  x  \  IND='  M  ' 

f'    ERROR  PERCENTAGES' 

*'       SS2y£  E5§  SEE         yStJE  N' 

7    2    r<:<2^0)/C13(  (  («»X2,C0.53    PE),C23    »E)»C23    *S),C23    ^ 

r '     ' 

JUMP4  J  y  •         ' 

-fCRSLOOP 


Figure   A.  6    (continued) 
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INPUT 
COURSE  OS 


PROMPT 


0 


3601  3602 


n 


*  see  COURSE  listing  for  details 
of  the  prompts 


Figure  A. 7    Input  Syntax  :  COURSE 
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1 

COURSE   : 

3601 

"82.2 

0 

3601 

14 

8 

6 

"83.2 

0 

3601 

14 

16 

"2 

"84.2 

365 

3601 

2 

0 

2 

"84.2 

525 

3601 

20 

22 

"2 

"83.2 

374 

3601 

4 

5 

"1 

"83.2 

525 

3601 

10 

11 

"1 

"84.2 

0 

3601 

25 

26 

"1 

"84.2 

374 

3601 

3 

4 

"1 

"83.4 

0 

3601 

14 

23 

"9 

"83.4 

365 

3601 

3 

11 

"8 

'82.4 

0 

3601 

14 

21 

"7 

"83.4 

374 

3601 

2 

5 

"3 

"83.4 

525 

3601 

9 

*7 

2 

GROUP 

MEAN 

VARIANCE 

N    (OF 

ERROR 

TERM  ) 

2.0 

".6 

2.3 

5 

.0 

4.0 

"3.0 

25.0 

3 

.0 

overall; 

"1.5 

10.0 

8. 

0 

COURSE   J 

3602 

"82.1 

0 

3602 

7 

11 

"4 

"83.1 

0 

3602 

7 

7 

0 

"83.1 

374 

3602 

1 

1 

0 

"83.1 

525 

3602 

6 

6 

0 

"84.1 

0 

3602 

12 

12 

0 

"84.1 

374 

3602 

3 

3 

0 

"84.1 

525 

3602 

9 

9 

0 

"83.3 

374 

3602 

3 

5 

"2 

"84.3 

374 

3602 

2 

4 

"2 

"84.3 

525 

3602 

12 

10 

2 

"82.3 

0 

3602 

15 

14 

1 

"83.3 

0 

3602 

15 

16 

"1 

"83.3 

686 

3602 

1 

0 

1 

"83.3 

525 

3602 

11 

11 

0 

"84.3 

0 

3602 

14 

14 

0 

GROUP 

MEAN 

VARIANCE 

N    (OF 

ERROR 

TERM) 

1.0 

.0 

.0 

4 

.0 

3.0 

".2 

3.2 

5 

.0 

OVERALL ; 

".1 

1.6 

9. 

0 

Figure  A. 8    Output  from  Input  Sequence  shown  in 
Figure  A. 7  :  COURSE  Program. 
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**•   Curricula  Related  Work 

In  looking  at  the  curricula  information,  we  want  to 
find  out  just  how  good  the  figure  'twelve'  is  for  the 
expected  input  from  the  Antiwhale  Warfare  curriculum.  Since 
curriculum  374  will  be  administering  the  program,  we  will 
use  the  data  from  374  as  an  approximation  of  curriculum 
449. 5's  performance.  Again,  the  variable  'OS'  contains  the 
information  we  want.  The  'CURRICP'  program  most  easily 
shows  this  data.  Through  proper  input,  we  can  call  up  the 
historical  performance  of  curriculum  374.  The  'CURRICP' 
program  provides  all  the  prompts  necessary  (see  figure  A. S) . 
Since  we  are  only  interested  in  one  curriculum  but  two  are 
required  for  proper  program  execution,  a  dummy  curriculum 
(333)  is  requested.  The  program  output  is  shown  in  figure 
A. 10   See  figure  A. 11  for  a  program  listing. 

As  we  can  see  from  figure  A. 10,  curriculum  374 
routinely  appears  to  overestimate  the  number  of  students 
expected  for  any  given  course.  To  show  more  adequately 
patterns  of  concern  in  the  data  (such  as  performance  for 
OS3601  and  OS3602) ,  we  can  look  at  the  •CUEHIC*  program 
sorted  by  course.  This  will  break  out  the  predictions  to 
better  show  how  well  curriculum  374  performs  for  individual 
courses.  Figure  A. 12  shows  the  inputs,  figure  A. 13  the 
outputs,  and  figure  A. 14  the  program  itself. 

As  we  can  now  see  from  the  course  data  shown  in 
figure  A. 13,  curriculum  374  generally  overstates  the 
expected  number  of  students  attending  courses  OS3601  and,  to 
a  lesser  extent,  OS3602.  We  can't  observe  any  possible 
patterns  by  quarter  due  to  the  limited  number  of  observa- 
tions. A  brief  look  at  the  error  percentage  table  shows  no 
predictive  errors  for  either  course  (they're  taught  or  not 
taught   when  they   say   so  -   no  'zeros'   in   the  actual  or 
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predicted  columns),  aDd  an  improvement  in  accuracy  from  3601 
to  36C2  (0%   error  free  to  50%  error  free  respectively) . 

In  summary,  through  use  of  the  curricula  related 
programs  we  have  found  that  a  slight  overestimation  of 
students  expected  is  usual  for  these  courses.  This  trend  may 
or  may  not  hold  for  increased  class  sizes  (of,  say,  twelve 
students  vice  three  or  four) .  Therefor,  combined  with  the 
information  we  previously  obtained  from  the  course  related 
programs,  we  conclude  that  new  sections  will  probably  not  be 
required. 


INPUT         PROMPT 
CUERICP  OS 


374  333 


*  see  COURSEP  listing  for  details 
of  the  prompts 


Figure  A-9    Input  Syntax  :  COREICP 
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CURRICULUM 

t  374 

"83.4 

374 

2101 

"83.4 

374 

2103 

"83*2 

374 

2103 

"83.3 

374 

3604 

"83.4 

374 

3601 

"83.3 

374 

3602 

"83.4 

374 

3603 

"84.1 

374 

3604 

"84.3 

374 

3602 

"83.2 

374 

3601 

"83.2 

374 

3604 

"84.1 

374 

2103 

"84.1 

374 

3401 

"84.2 

374 

2103 

"84.2 

374 

3601 

"84.3 

374 

3604 

"83.1 

374 

3602 

"83.1 

374 

3604 

"84.1 

374 

3602 

"84.1 

374 

3603 

0 

2 
1 
1 
2 
3 
4 
r> 

*— 

2 
4 
0 
0 
0 
2 
3 
2 
1 
4 
3 
1 


6 
6 
4 
4 
5 
5 
6 

4 

4 
5 
1 
1 
1 
3 
4 
3 
1 
4 
3 
1 


'6 
"4 
'3 
"3 
■3 
•  n 

■  o 

■  2 

"3 


0 
0 
0 
0 


CURRICULUM   J   333 


Figure  A. 10    Output  froa  Input  Sequence  shown  in 
Figure  A. 9  :  CDBEIC?  Program. 
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v    curricp  matrix ;x; i ;y;z 

C13   ft 

121  ft   THIS   FUNCTION,   GIVEN   AN   INPUT   MATRIX   IN   THE   FOLLOWING   FORM   • 

C33   a 

C4]    A         QTR    CURRIC    CSE    ACTUAL    PREDICTED    ( ACTU AL -PR ED  I C TE D ) 

cs:  « 

£63    A   WILL   SORT   AND  DISPLAY   THE  DATA  BY  CURRICULUM   IN  THE 
C73     A   FOLLOWING   MANNER   J 

cs3  a 

Z92  ft         CURRICULUM 

cio:  A 

[H]   a  DATA  ORDERED  BY  MAGNITUDE  OF  ERROR  TERM 

C12D  ft 

C133  A  («1-L  QUARTERS  WILL   BE   DISPLAYED  AS   NEGATIVE  NUMBERS,   THIS   ALLOWS 

C143  ft  F0R   CORRECT   PLACEMENT   OF   THE  OUTPUT   COLUMNS  WITHOUT  RESORTING 

C153  ft  T0  COMPLEX   CHARACTER  ARRAY   MANIPULATIONS.) 

:i63  a 

C173  ft   THE  GOAL   OF  THIS  PROGRAM   IS  TO  DISPLAY  OBVIOUS  PATTERNS  THAT 

C183  ft  MftT  THEN  BE  LOOKED  AT  MORE  CLOSELY  USING  THE   'COURSE'   OR   'CURRIC 

C193  ft  FUNCTIONS,   CURRICULA  MAY  BE  SELECTED  INDIVIDUALLY   IF  SO  DESIRED, 

C203  ft 
C213    MATRIX«-MATRIXX  (  f  MATRIX  )  f      "111111 

C223   f'iF  tou  want  all  curricula  displayed,  enter  i' 

C233    t'OTHERMISE,   ENTER  0  TO  SELECT  SOME  SUBSET  OF  THE  CURRICULA' 

C243    -»JUMPlx\Q 

C253    t'THE  FOLLOWING  CURRICULA  ARE  AVAILABLE  FOR  SELECTION   •' 

C263   txt.x[^)<fUHiauE  matrixl;233 

C273   t'Emter  the  curric  numbers  you  want,  enter  x  to  get  them  all , ■ 

C283    f ' (NOTE   ;   YOU   MUST  ENTER  MORE   THAN  ONE  CURRICULUM) ' 

C293    TffXfQ 

C303   4JUMP2 

[313  JUMPJ  J  Y«-f  X«-XL4X4-UNIQUE  matrix^  ;  23  3 

C323  jump2:i«-i 
C333  CRCLOOPit'  ' 

:343  -»ox\i>y 

C353    (^'CURRICULUM   ;   '),tX[I] 

C363   z«.(XLil»MOTRixLj23)/Cia  matrix 

C373    r2  ORDER  £ 

C383   i«-i  +  l 
C393   r'  ' 
C403   *'  ' 

C413    -+CRCLOOP 
7 


Figure   A- 1 1         'CURRICP'    Program 
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INPUT 
CUEEIC  OS 


FEOMP1 


0 


374  333 


*  see  COUESE  listing  for  details 
of  the  prompts 


Figure  A.  12    Input  Syntax  :  CUEEIC 
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CURRICULUM   ;   374 

■   I 

"33.4 

374 

2101 

0         6 

"6 

"83.4 
"33.2 
"34.1 

374 
374 

374 

2103 
2103 
2103 

2         6 
1         4 
0         1 

"4 

"3 

"1 
"1 

"34.2 

374 

2103 

2        3 

"34.1 

374 

3401 

0        1 

"1 

"83.4 
"83.2 
"34.2 

374 
374 
374 

3601 
3601 
3601 

2  5 
4        5 

3  '  4 

"3 

"1 
"1 

"33.3 

374 

3602 

3        5 

_-i 

"84.3 

374 

3602 

2        4 

~2 

"83.1 

"34.1 

374 

374 

3602 
3602 

1         1 
3        3 

0 
0 

"33.4 

374 

3603 

4         6 

~ i 

"84.1 

374 

3603 

1         1 

0 

"83.3 

374 

3604 

1         4 

-3 

"84.1 

374 

3604 

2         4 

—  "7 

"83.2 

374 

3604 

0         1 

"1 
"1 

0 

"84.3 

"83.1 

374 
374 

3604 
3604 

2        3 

4         4 

GROUP 

MEAN 

VARIANCE    N 

(OF  ERROR 

TERM) 

2101.0 

"6.0 

1.0 

1.0 

2103.0 

"2.3 

2.3 

4.0 

3401.0 

"1.0 

1.0 

1.0 

3601.0 

"1.7 

1.3 

3.0 

3602.0 

"1.0 

1.3 

4.0 

3603.0 

"1.0 

2.0 

2.0 

3604.0 

"1.4 

1.3 

5.0 

OVERALL   : 

"1.7 

2.3 

20.0 

ERROR   PERCENTAGES 

GROUP 

PRE 

£££ 

NONE 

H 

2101.00 

1.00 

.00 

.00 

1  .00 

2103.00 

.25 

.75 

."00 

4.00 

3401 .00 

1.00 

.00 

.00 

1 .00 

3601.00 

.00 

1.00 

.00 

3.00 

3602.00 

.00 

.50 

.50 

4.00 

3603.00 

.00 

.50 

.50 

2.00 

3604.00 

.20 

.60 

.20 

5.00 

CURRICULUM 

:  333 

GROUP 

MEAN 

VARIANCE    N 

(OF  ERROR 

TERM) 

OVERALL   ; 

1.0 

.0 

.0 

ERROR  PERCENTAGES 

GROUP 

ppg 

ace 

NONE 

H 

Figure  A. 13    Output  from  Input  Sequence  shown  in 
Figure  1.12  :  CURBJC  Program. 
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?CURRIC[|]]V 

<?    curric    MflTRi;<j!<;:)Y)ZjMC)vc)JiX2)T2;Majvo)WjAjHs;HE;iND(aE)PEjV)Nc;nQ 

[I]  ft 

C23  ft   THIS   FUNCTION,   GIVEN   AH   INPUT   MATRIX   IN   THE   FOLLOWING   FORM   J 

[3]  ft 

[4]  n         OTR    CURRIC    CSE    ACTUAL    PREDICTED    (ACTUAL-PREDICTED) 

[5]  ft 

[6]  ft   WILL  SORT  AND  DISPLAY   THE   DATA   BY  CURRICULUM   IN   THE 

[7]  ft  FOLLOWING  MANNER   J 

[8]  ft 

[9]  A        CURRICULUM 

CIO]  A                                   QUARTER   1   DATA          COURSE   1   DATA 

[II]  ft  quarter  2  bata  or   course  2  data 

[123  ft                                   QUARTER   3   DATA                     , 

[13]  ft                                   QUARTER   4   DATA          COURSE  H  DATA 

[143  ft                MATRIX  OF  GROUP   MEANS,   VARIANCES  AND  N 

[153  ft                OVERALL   MEAN,   VARIANCE   AND  N 

[163  ft 

[17]  ft   a  MATRIX   OF  ACCURACY,   PREDICTIVE  AND  NO  ERROR  PERCENTAGES   IS 

[183  ft  AVAILABLE  IF  DESIRED, 

[193  ft 

[203  ft   (ALL  QUARTERS  WILL  BE  DISPLAYED  AS  NEGATIVE  NUMBERS,   THIS  ALLOWS 

[213  ft    FOR  CORRECT  PLACEMENT   OF   THE  OUTPUT   COLUMNS  WITHOUT  RESORTING 

[223  ft    TO  COMPLEX   CHARACTER   ARRAY   MANIPULATIONS,) 

[233  n 

[243  ft  CURRICULA  MAY  BE  SELECTED  INDIVIDUALLY   IF  SO  DESIRED, 

[253  « 

[263  MATRIX4-MATRIXX  (  f  MATRIX  )  f   "111111 

[273  t1 IF  TOU  WANT  ALL  CURRICULA  DISPLAYED,   ENTER   I1 

[283  t ' OTHERWISE ,   ENTER  0  TO   SELECT  SOME   SUBSET  OF   THE  CURRICULA' 

[293  4JUMP1X1Q 

[303  r  '  THE  FOLLOWING  CURRICULA  ARE  AVAILABLE  FOR  SELECTION   •' 

[313  fX«-x[*x«-UNiauE  matrix[;233 

[323  t'EHTCR   THE  CURRIC  NUMBERS   YOU  WANT,   ENTER  X   TO  GET   THEM  ALL,' 

[333  t'(HOTE  ;  you  must  enter  more  than  one  curriculum)' 

[343  i-<-fX4-a 

[353  -»Jump2 

[363  Jumpi  •  Y«-px«-x[4x«-uNiouE    matrix[;233 

[373  JUMP25K-1 

[383  t'THE   DATA  MAY   BE  DISPLAYED   (TO  LOOK   AT  PATTERNS)   IN   GROUPINGS   BY' 

[393  f'EITHER  QUARTER  OR  COURSE,    FOR  QUARTER  GROUPINGS,   ENTER   1   }   FOR' 

[403  ^'COURSE   GROUPINGS,   ENTER   3,' 

[413  v«-a 

[423  t'ACCURACT  ERROR,   PREDICTIVE  ERROR   AND  NO  ERROR  PERCENTAGE  MATRIX?' 

[433  *•'  ENTER   ''  OR  N  • 

[44]  IND«-a 

[453  CPCLOOP'f'   • 

[46]  ->0xii>T 

[47]  ( r ' curriculum    •     ')ltx[i] 

[483  z«-(><[i3=*«TRix[;23>/r.l3    matrix 

[49]  MCt-MEAN  z[;6] 

[503  vc«-vapia  2[;63 

C513  '-'Ct-pzcj^] 

[52]  ->«-l 

[53]  4NEXTX»V=3 

[54]  «<-10x(-zCfV])_L-zL;v] 

C55]  +out 

[56]  next:a«-z[  jv;j 

[57]  out  •  Y2*-pX2*-!<2[4>:2«-UHIOUE  °] 

[58]  MS(-AE(.PE(-imfMQ(.VQflO 

[59]  QTRLOOP  <  -»ENDx  \  J)  Y2 


Figure  A.  14    'CUEfilC  Program 
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cao: 

[62: 
C633 
Z641 
L651 

:66: 

C67] 

[68] 

C703 
C713 
C723 
C73] 
C74D 
C753 
C763 
C773 
S78] 
C79] 

C80: 

cai: 

[82] 
[83] 
[84: 


w«-(  (>: 

«a  i-mo 

1 1  a «-  m  a 

•»JUMP 
PE«-PE 
AE«-AE 
M  SfHS 
JUMP3: 

r '  ' 
J«-J  +  l 

-»OTRL 
END*  I«- 

r'   s 

7  1  t 
r '  ' 

(T'OV 

r '  ' 

-►JUMP 
*'    E 

t'   s 

7  2  ♦ 

T  ' 
JUMP4 • 

4CRCL 
7 


2CJ3=o)/:iD  z>  order  6 

,  MEAM  WQJ6] 
,VARIA  WCJ6] 
,HE«-fWCJ&] 
3X I IND= ' N< 

»  <+/ <*»C»4 3=0 )vt*C}  53=0 >*NE 

»<+/(WC»4]^0)a(wc;5]!<0)awc}6]j!0)-!-me 
f(+/(wc;4]^0)A(wc;5]?!0)AWc;6:=0)^ME 


OOP 

i  +  l 

ROUP       MEAN    VARIANCE    N    (OF   ERROR   TERM)' 

<<*x2iC0.5D  mq),c23  va>,c2:  "« 

ERALL   •   ')»   6  1   *MC,VC,MC 

4X l IHD= • N ' 

RROR   PERCENTAGES' 

5QUP       PRg       ACC      HONE       M' 

(  (  (S:<2»C0.53  pe),c23  ftE)»C23  '-<S)i[2:  "« 
f '  ' 

OOP 


Figure  A.  14  (continued) 
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D.   DATA  COLLECTION  AMD  MAINTENANCE 

To  provide  the  matrix  of  historical  data  to  be  displayed 
and  analyzed,  two  sources  are  used.  The  prediction  data 
comes  from  the  course  loading  forecast  delivered  to  the 
department  chairmen  (see  figure  A. 15).  The  easiest  way  to 
compile  that  data  is  to  obtain  a  copy  of  the  individual 
course  predictions  and,  on  the  forecast  itself,  change  the 
letter  codes  found  in  the  "S/D/T"  column  into  curricula 
numbers  (using  the  ceding  list  contained  in  appendix  B)  with 
associated  totals.  For  example,  the  totals  you  would  write 
in  from  figure  A. 15  for  OS3602  are  525:  9,  374:  3,  and  0 
(total) :  12.  The  ether  document  required  is  the  course 
grade  sheet  held  by  the  registrar  (see  figure  A.  16) .  This 
information  is  subject  to  the  privacy  act  and  requires  a 
written  reguest  for  permission  from  the  department  chairman 
involved  to  the  office  of  the  registrar  for  access  tc  the 
files.  The  records  themselves  are  kept  according  to 
academic  year.  This  notation  is  confusing;  to  prevent  prob- 
lems, ask  for  something  like  the  *  83-84  school  year'  data. 
These  records  are  not  allowed  to  leave  the  office,  and  at 
last  check  were  not  available  on  the  computer.  The  data 
must  be  recorded  by  hand.   A  convenient  format  is : 

Quarter   Course   Section   Total*   #Attending/Curricula 

Once  the  above  listed  data  have  been  collected  and  are 
in  front  of  you  and  a  computer  terminal,  a  CMS  file  as  shown 
in  figure  A. 17  can  be  created  by  combining  the  informaticn. 
The  columns  correspond  to  the  quarter,  curriculum,  course, 
actual  and  predicted  number  of  students.  The  CMS  file  thus 
created  can  be  transferred  to  APL  and  named,  for  instance, 
CSNEXT  through  the  following  command  sequence: 

(get  into  APL) 

) LOAD  990  CMSIO 
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)  KSID    HOLD 

OSNEXT    <=    CMSREAD 

(follow    prompts    to   read    your   CMS    file) 

) EEASE    CMSIO 
)  SAVE 

NOTES    :    1)    The    APL    variable    'OSNEXT'    now   exists 

on    your    disk    in    the    APL    workspace    'HOLD* 

2)    The    •<=•    character   shown   corresponds    to 
the   left-pointing    assignment    arrow   in    APL 

To   add      the    (Actual      minus    Predicted)         term    to      the    matrix, 
enter  : 

CSNEXT    <=    OSNEXT, [2]    0SNEXT[;4]    -    OSNEXT[;5] 

To    add    this      matrix    tc   the    variable    OS    (assuming      that    OS   is 
in    the    workspace),    enter: 

CS    <=    OS,[ 1  ]    OSNEXT 

The  entire   updating  procedure   should  take   about  one   full 
workday  for  each  year  of  data. 
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OFFICIAL    GRADE    ROSTER 

PAGE     1 

NAVAI 

POSTGRADUATE    SCHOOL 

MONTEREY, CALIFORNIA       93943 

DATE     13    APR 

COURSE    NO.    OS3602    SEGEMENT 

00 QUARTER 

1       AY    83-84 

COMBAT 

MODELS 

PROF. 

SHUDDE,     REX    H. 

3    LECTURE 

HCURS       1    LABORATORY 

HOURS 

CORPS/ 

NAME 

RANK 

COUNTRY 

CURR 

MARK 

1. 

xxxxxx. 

xxxx 

X 

LT 

USN 

#374 

A 

2. 

xxxxxx. 

xxxx 

X 

LCDR 

USN 

#374 

B 

3. 
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X 

LCDR 
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#525 

B 

4. 
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#525 
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#525 
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X 

LT 
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#525 

B 

1  1. 

xxxxxx, 

xxxx 

X 

LCDR 

USN 

#525 

B 

12. 

xxxxxx, 

xxxx 

X 

LT 

USN 

#525 

B 

FCBf 

I    4 

THIS 

;    IS    THE 

OFFICIAL 

GRADE 

ROSTER    TO 

BE    FILED    IN    THE 

ACAEEMIC    DEPARTMENT 

. 

i 

Figure  A. 16    Sample  Grade  Sheet  (names  removed) 
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Figure  A. 17   CMS  File  Format 
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E.   OTHER  AVAILABLE  PROGRAMS  AND  DATA 

A  complete  listing  of  the  functions  available  in  the 
•OSDATA*  workspace  is  shown  in  table  VII  .  The  COURSEP, 
COURSE,  CURRICP  and  CURRIC  programs  have  been  previously 
demonstrated.  The  orly  non-trivial  function  remaining  to  be 
shown  is  the  ' WILCOXSR*  program  (see  figure  A. 18) .  As  shown 
in  the  figure,  this  program  is  based  on  the  Wilcoxian  Signed 
Rank  Testn  detailed  in  [Ref-  5  pp  280-288]. 

A  self-descriptive  listing  cf  the  variables  contained  in 
the  workspace  is  shown  in  table  VIII  .  The  latest  data  pres- 
ently available  is  the  spring  of  1984  (academic  year  83-84) 
figures. 


11Eor  a  complete  description  of  the  various  hypotheses 
that  may  be  explored,  see  [Ref.  5  pp  280-288],  In  the  body 
of  the  thesis,  the  magnitudes  of  the  forecasting  errors  (y) 
were  tested  against  the  magnitudes  of  prediction  errors 
derived  from  course  matrices  (x) .  The  hypothesis  tested 
(with  rejection  the  desired  result)  was  that  the  *x'  values 
tend  to  be  egual  to  or  larger  than  the  '  y'  values.  This  was 
one  of  six  possible  interpretations  of  the  null  hypothesis. 
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TABLE  VII 
Functions  Available 

Function   Syntax        Description 

********   ********      ****************************** 

CCUESFF  COUBSEP  mtx  (previously  shown) 

CCUBSE  COURSE  mtx  (previously  shown) 

CUBEICP  CUEEICP  mtx  (previously  shown) 

CUEBIC  CUBEIC  mtx  (previously  shown) 

WIICOXSE   y  WILCOXSE  x   Shows  the  Wilcoxian  Signed 

Rank  Test  results  (see 
description  in  text) 

OEDEE      A  OEDEE  E      Orders  rows  of  matrix   A 

according  to  decreasing  mag- 
nitude of  column  B,  returning 
the  ordered  matrix 

UNIQUE     UNIQUE  A     Eeturns  the  unigue  elements 

of  the  vector  or  matrix  A  in 
ascending  order 

MEAN       MEAN  X       Eeturns  the  arithmetic  mean 

of  vector  X 

VAEIA      VABIA  X      Eeturns  the  unbiased  estimate 

of  the  variance  of  vector  X 
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v    r    wilcoxsr  :<;d;t;r 
C13   « 

C23  ft   THIS  FUNCTION  PROVIDES   THE  HIILCOXIAN   SIGHED  RANK   TEST 

C3.3  ft   STATISTIC   (FROM  COHOVER  PG,   280).   INPUTS   :<   AND   l   ARE 

C4]  a,      DATA  VECTORS   OF  EQUAL.  LENGTH, 

C51  ft 

C63      D«-(D*0)/D«-T_X 

C71      R«-(RANK  |  D)  xBt|  D 

C3D    T<-(^/R)i(+/R*2)  *0.5 

C93      (r'TEST  STATISTIC   =   '),tT 
7 


Figure  A.  18        Uilcoxian   Signed   Rank   Test   Program 
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TABLE    VIII 
Variables    Available 

VAEIAELE      SHAPE       DESCRIPTION 
********    *******    ************ 

OS82  69x6         AI    (Academic    year)     81-82    data; 

includes    summary    information    only 
(no   data   for   individual   curricula) 

CS83  255x6         AY   82-83    data;    complete    listing 

OSTRK83      140x8         AY    82-83    non-entry    course   data.    The 

additional  columns   contained    in    this 
matrix    (columns    6   and    8)    correspond 
tc   an    'adjusted    prediction'    and    an 
'adjusted    error'    term-    derived    from 
the   course    matrices.    This  array    was 
used    to    check    the    feasability    of 
making    adjustments  based   solely    on 
the    'track'    a    student    takes   using    the 
Wilcoxian    Signed   Rank    Test    previously 
mentioned. 

OS84  228x6         AY    83-84    data;    only    gtrs    1,2    and    3 

CS  552x6        Combination   of   all   of    above 

(without    repetition) 

OA83  113x9         This    matrix   is    set    up   in   a    different 

format    from   those   previously    dis- 
cussed.      The    nine   columns    represent 
Quarter,       OA    course    number,       Total 
attending,       USN      attending,         USA 
attending,      OSMC   attending,      Others 
attending,    Total    predicted,    and    Total 
attending      minus      total    predicted, 
respectively.       The    original  intent, 
dropped   due   to   time   constraints,    was 
tc   use   these    figures    to    aid   in    OA 
elective  scheduling    by    looking   at 
expected    number    of    upcoming    free 
electives    by   service   through    use    of 
the   OA   course   matrix. 
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APPENDIX  B 
SCHEDULING  CODES 

This  appendix  contains  an  alphabetical  listing  of  the 
specialty  codes  used  in  the  scheduling  process  as  of  August 
1984.  This  listing  cnly  uses  the  first  two  letters  of  any 
individual  coding  because  the  remaining  letters  specify 
either  the  degree  awarded  or  the  degree  and  month  of  input, 
neither  cf  which  change  the  curriculum  associated.  Using 
these  codes,  numbers  of  students  predicted  by  any  given 
curriculum  for  any  given  course  can  be  taken  from  the  indi- 
vidual course  forecast  promulgated  by  the  programming 
office.  Code  030  1  at  extension  2773  at  the  Naval 
Postgraduate  School  can  be  contacted  for  updated 
information. 

Specialty 
Curricula   Degree 
Numbers    Codes     Program 
*************************************************  ***** 

610  AC —       Aero  Engr,  incl  Engr  Sci 
AT — 

611  AX —       Aeroelectronics 

368  CS —  Computor    Science 

600  DC —  Electrical    Engineering    (Comms) 

Note:    DCTR   is   curriculum    460 

590  EC —  Electrical    Engineering    ;    various 

ED —  sub-specialties.    Note:    EETR    is 

EG —  curriculum    460 
EE — 

380  GH —  Math   option    of    advanced    science 
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381 

GP 

620 

HC 

HM 

Specialty 

Curricula   Degree 

Numbers    Codes     Program 
****************************************************** 

Ehysics  option  of  adv.  science 
Communications  Management 

525      IX —       antisubmarine  Warfare   Note:  IETR 

is  curriculum  460 

365      JM--  Command,  Control  and  Communica- 

JN —  tions  ;  various  sub-specialties 

JO —  Note:  JCTR  and  JXTR  are  from 

JP —  curricula  460 
JS — 

595       KE--       Electronic  Warfare  Systems  Tech- 
nology  Note:  KETR  is  curric  460 

Admin  Sci  -  Econ,  USMC 

Admin  Sci  -  Financial  mgt 

Admin  Sci  -  Human  Resource  mgt 

Admin  Sci  -  Inventory 

Admin  Sci  -  International  stu. 

Admin  Sci  -  Material 

Admin  Sci  -  Ops  Research,  Army 

Admin  Sci  -  Personnel 

Admin  Sci  -  Procurement 

Admin  Sci  -  MGMT  Sci,  Coast  Guard 

Admin  Sci  -  Transportation 
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817 

ME- 

837 

MF 

857 

MH- 

819 

MI 

817 

MK 

827 

MM 

817 

MO 

847 

MP 

815 

MR- 

817 

MS 

814 

MT 

Specialty 
Curricula   Degree 
Numbers    Codes      Program 
****************************************************** 

8  13      MV —       Admin  Sci  -  Material  Movement 

817       MX —       Admin  Sci  -  Civ.  Federal  Program 

570      NE —  Mechanical  Engineering 

NF —  ME  via  460 

NI —  ME  via  460 

NR —  ME  via  460 

NT —  ME  via  46  0 

441  OH —  Hydrographic  Science 

440  OS —  Oceanography   Note:  OCTE  is  460 

367  PL —  Information  Systems  (from  CS ) 

360  RA —  Cps  Analysis,  Army  ' 

460  EC —  Engr  Sci  for  Ops  Analysis 

360      EG —  Cps  Analysis,  General 
EN —       Navy 
ES —       Supply 
EP —       PHD  program 

591       SE —       Space  Systems  Engineering 

366       SO —       Space  Systems  Operations   Note: 

SXTE  and  SOTE  are  curriculum  460 

535      UX —       Underwater  Acoustics   Note:  UETE 

is  curriculum  460 
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Specialty 
Curricula   Degree 

Numbers    Codes  Program 
****************************************************** 

530      FA —  Weapons  Systems  Technology 
WC — 

WE — 
WM — 

532       WN —  Weapons  Systems  Science 

530  flP —  Weapons  Systems  Technology 

531  WS —  Weapons  Systems  Science 

372  XA —  Meteorology 

373  XS —  Met.  and  Oceanography 

374  XT —  Air-Ocean  Tactical  Env.  Support 

Note:  XOTS  and  XXTR  are  460 

683  YA —  NSA  -  USSR,  Europe;  USN 

682       YB —  NSA  -  Asia/Pacific;  USN 

68  1       YC —  NSA  -  Middle  East,  Africa;  USN 
YD —  -  S.  Asia;  USAF,  USA 

682       YE —  NSA  -  Asia/Pacific;  USAF,  USA 

681       YF —  NSA  -  S.  Africa;  USAF,  USA 
YG —  -  N.  Africa;  USAF,  USA 

YM —  -  Middle  East;  USAF,  USA 

687       YN —  NSA  -  Nuclr/Plng;  USN 

684  YO —  NSA  -  Intl/Org/Neg;  USN 
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Specialty 

Curricula   Degree 

Numbers    Codes     Program 


686 

YP — 

NSA 

683 

YR — 

NSA 

825 

YS — 

NSA 

683 

Ytf — 

NSA 

Strat/Plng;  USN 
DSSR/E.  Europe 
Navy  Intell;  USN 
OSSE/W.  Europe 
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APPENDIX  C 
DETAILED  SUMMARY  OF  RESULTS 

This  appendix  contains  a  brief  description  of  the 
results  found  for  each  course  and  curriculum.  These  results 
are  listed  in  ascending  numerical  order  first  for  the 
courses  then  the  curricula.  Courses  or  curricula  that  cause 
problems  are  flagged  with  a  'P*  in  front  of  the  identifying 
number. 

Course 

Number   Comments 

P  210  1    Not  taught,  only  predicted. 

P  2102    Large  magnitude  errors,  generally  underestimates. 

Curriculum  590  causes  most  of  the  problems,  curric- 
ulum 600  to  a  lesser  extent.  No  quarterly  pattern. 

P  2103   Generally  gocd  except  for  curriculum  531  which 

never  sends  people  but  has  predicted  40,  27  and  7. 
No  significant  quarterly  pattern. 

P  3001    Prediction  errors  in  third  and  fourth  quarters  - 
not  predicted  but  taught.  Only  curriculum  368. 

3002  Generally  gocd.  Only  taken  by  curriculum  825  though 
others  are  predicted.  Need  more  data. 

3003  Generally  good.  Need  more  data. 

3004  Generally  gccd. 

3005  Generally  good.  Need  more  data. 
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Course 

Number   Comments 

******   *************************************************** 

3006   Generally  gcod  except  for  curriculum  817.  Only 
taught  first  and  third  quarters.  Overall  class 
sizes  fairly  stable  at  63  and  87  respectively 
(three  sections  required)  . 

P  3008  Definite  prediction  problem  that  appears  to  be 
improving.  Curriculum  365  provides  most  of  the 
input. 

P  310  1    Curriculum  460  not  predicted  generally  causing  a 
significant  underestimation  (usually  results  in 
adding  an  additional  section) .  More  data  needed. 

P  3  102    Overall  significant  overestimation  from  cumulative 
effect  of  8XX  curricula.  Worst  in  third  quarter. 

P  3103   Overall  significant  overestimation  from  cumulative 
effect  of  8XX  curricula.  No  quarterly  trend. 

P  3104   large  errors  in  summary  predictions  generally  due 
to  prediction  vice  accuracy  errors.  All  curricula 
except  570  and  817  experience  these  errors.   First 
quarter  predictions  look  good,  underestimates  occur 
in  the  second  quarter,  overestimates  in  the  third. 

P  3105   Significant  prediction  problems.  Generally  offered 
only  third  quarter. 

3302  Not  enough  data. 

3303  Generally  gcod.  525  provides  majority  of  input. 

3401  Not  enough  data. 

3402  Prediction  problem,  not  severe. 
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Course 

Numfcer   Comments 

******   *************************************************** 

3403   Generally  excellent,  not  enough  data. 

P  3404  Curricula  365  and  825  provide  most  of  the  inputs 
to  this  course.  365  does  an  excellent  job,  825  a 
poor  one. 

3601  Generally  gccd. 

3602  Generally  excellent. 

3603  Generally  excellent. 

3604  Generally  good. 

P  3702  Significant  prediction  problem.  Not  taught  first 
quarter,  only  predicted.  Curriculum  847  provides 
most  of  the  input. 

4601  Generally  excellent. 

4602  Generally  excellent,  needs  more  data. 
4701    Generally  gocd. 


************************************************************ 

**  END  OF  COURSE  LISTING  ** 

************************************************************ 
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Curriculum 

Numter  Comments 

P     0  Summary  predictions  are  innaccurate  due  to  course 
effects.   No  quarterly  trend  seen. 

p   360  Seldom  predicted  for  OS  courses.  Minor  effect  due 
to  small  number  of  inputs. 

365  Generally  excellent,  possible  problem  with  OS3008 

366  Generally  excellent. 

367  Generally  good,  need  more  data  in  OS3101- 

P   368  Third  quarter  prediction  problem  developing. 

372  Not  enough  data. 

373  Not  enough  data. 

P      374  Tends    to    overestimate. 

380  Not    enough  data. 

440  Not  enough  data. 

441  Not  enough  data. 
P   46  0  Never  predicted. 

525  Generally  excellent. 

P   530  Problem  in  OS3104. 

P   531  Significant  problems  affecting  OS2103  and  0S3104. 

532  Not  enough  data. 

535  Not  enough  data. 

570  Possible  errors  in  third  quarter,  need  more  data- 
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Curriculum 

Numker    Comments 

******    ************************************************* 

P      590  Problem    in  OS2102 

591  Looks   good,    not  enough   data. 

600  Generally    good. 

610  Not  enough  data. 

611  Not  enough  data. 
620     Generally  good. 
683     Not  enough  data. 

686  '  Not  enough  data. 

687  Not  enough  data. 

8  13     Generally  excellent. 

8  11     Generally  excellent. 

815     Generally  fair. 

P   817     Problem  with  OS3006,  good  otherwise. 

P   819      Various  prediction  problems. 

P   825     Problem  with  OS3404. 

P   827     Overestimates  for  OS3102  and  OS3103. 

P   837     Problem  in  OS3102,  prediction  error  for  OS3105. 

P   847     Problem  with  OS3702.  Good  with  OS3103.   Third 
quarter  predictions  are  poor. 

P   857     Routine  innaccuracies  with  no  apparent  pattern. 

999     Insignificant  input. 
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APPENDIX  D 
CONDENSED  COO'BSE  MATRICES 

This  appendix  contains  the  condensed  course  matrices  for 
all  curricula  that  take  OS  courses.  Curricula  610,  611,  681, 
682,  683,  and  684  dc  not  have  any  OS  courses  required  and 
are  not  included-  ' xxxxxx'  entries  in  the  condensed  matrices 
indicate  that  the  students  are  onboard  but  not  taking  OS  or 
OA  designated  courses  (these  entries  are  not  made  following 
the  final  OS  or  OA  course  taken  to  prevent  display  clutter). 
The  entries  are  based  on  the  printed  course  matrices  as 
published  as  of  August,  1984.  These  matrices  were  provided 
by  the  individual  curricular  offices. 
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APPENDIX  E 
ENTRY  /  TRACK  COOBSE  DESIGNATIONS 

This  appendix  contains  a  complete  listing  of  the  OS 
courses  divided  into  'Entry1  and  'Follow-on*  catagories.  An 
'Entry*  course  is  the  first  OS  course  that  is  taken  in  a 
given  course  matrix  (for  a  given  curriculum)  ;  the  first  time 
a  curriculum  can  be  'seen*  by  the  Operations  Analysis 
Department.  'Follow-cn'  courses  are  those  courses  that  are 
taken  in  guarters  following  the  guarter  the  'Entry'  course 
is  taken.  All  curricula  take  one  of  the  six  'Entry*  courses 
prior  to  any  further  OS  course  work  with  the  exceptions  of 
curricula  610,  611,  681,  682,  683  and  684  which  have  no 
reguired  OS  courses. 

ENTRY  TRACK  (Follow-on) 

COURSE   CURRICULUM   COURSES,  All  OS 

******   **********   ******************************* 

OS2103  365  3604,3404,3008,3603,4602 

366  3604,3404,3008,3603,4602 

374  3601,3602,3603,3604 

525  330  3,3604,36  01,3401,460  1,360  2 

535  none 

595  3604,3003,3603,3403,4601 

CS3101  367  3004 

620  3005 

686  none 
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ENTRY  TRACK    (Follow-on) 

COURSE      CURRICULUM       COURSES,     ALL    OS 

♦  ♦#♦♦♦        ♦♦♦♦♦♦♦♦♦♦         ♦  ♦♦  ♦♦  ♦  ♦  ♦  ♦  ♦  ♦  ♦  ♦  A  £  ♦  ♦♦  ♦  ♦  ♦  ♦  ♦  ♦  ♦  4s  ♦  ♦♦  ♦  ♦ 

CS310  1      687  none 

825  3404,3002 

OS3001      368  none 

OS3  104     372  none 

373  none 

440  none 

441  none 

530  none 

531  none 

532  cone 
570  none 

CS2102      590  3104 

591  none 

600  none 


CS3102 

813 

3103,3006 

814 

3103,3006 

815 

3103,3006 

817 

3103,3006,3604 
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ENTRY  TEACK    (Follow-on) 

COURSE       CURRICULUM       CCURSES,     ALL    OS 

$$$$$$  $$  ^c  *  #4:  £  %  $  •&  $  ^  *  $#  $  4:  *  jj;  #  $  $  3}t  +  $  *  ^t  #  ^t  %  #  $  £  $  £  ajc  $  #  £   ajr  £ 


OS3102 

819 

3103,3006 

827 

3103,3006 

837 

3103,3006 

847 

3103,3006,3702 

857 

3103,3006 
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